[jira] [Commented] (HBASE-12234) Make TestMultithreadedTableMapper a little more stable.
[
https://issues.apache.org/jira/browse/HBASE-12234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168029#comment-14168029
]
Hadoop QA commented on HBASE-12234:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12674340/0001-HBASE-12234-Make-TestMultithreadedTableMapper-a-litt.patch
against trunk revision .
ATTACHMENT ID: 12674340
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11312//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11312//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11312//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11312//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11312//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11312//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11312//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11312//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11312//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11312//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11312//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11312//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11312//artifact/trunk/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11312//console
This message is automatically generated.
Make TestMultithreadedTableMapper a little more stable.
---
Key: HBASE-12234
URL: https://issues.apache.org/jira/browse/HBASE-12234
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0, 2.0.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Attachments:
0001-HBASE-12234-Make-TestMultithreadedTableMapper-a-litt.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12235) Backport to 0.94: HBASE-9002 TestDistributedLogSplitting.testRecoverdEdits should test correct region
[
https://issues.apache.org/jira/browse/HBASE-12235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-12235:
Description:
I had TestDistributedLogSplitting.testRecoveredEdits fail on a internal test
rig. It's very infrequent.
The proximal failure looks like HBASE-9002, the assertion finds 0 edits instead
of 1000.
Looking through the logs, the #regions printed is 1
{code}
line 236LOG.info(#regions = + regions.size());
{code}
And then in the makeHLog method I never see any log entry for the actual writing
{code}
line 569for (int i = 0; i n; i++) {
line 570 LOG.info(region + hris.get(i).getRegionNameAsString() + has
+ counts[i] + edits);
line 571}
{code}
I think what's happening is we're picking a region that is META or ROOT, so the
checks after the #regions= line removes the region and we end up inserting no
data.
It looks like the patch for HBASE-9002 should fix this.
was:
I had TestDistributedLogSplitting.testRecoveredEdits fail on a internal test
rig. It's very infrequent.
The proximal failure looks like HBASE-9002, the assertion finds 0 edits instead
of 1000.
Looking through the logs, the #regions printed is 0
{code}
line 236LOG.info(#regions = + regions.size());
{code}
And then in the makeHLog method I never see any log entry for the actual writing
{code}
line 569for (int i = 0; i n; i++) {
line 570 LOG.info(region + hris.get(i).getRegionNameAsString() + has
+ counts[i] + edits);
line 571}
{code}
I think what's happening is we're picking a region that is META or ROOT, so the
checks after the #regions= line removes the region and we end up inserting no
data.
It looks like the patch for HBASE-9002 should fix this.
Backport to 0.94: HBASE-9002 TestDistributedLogSplitting.testRecoverdEdits
should test correct region
-
Key: HBASE-12235
URL: https://issues.apache.org/jira/browse/HBASE-12235
Project: HBase
Issue Type: Task
Components: test
Affects Versions: 0.94.0
Environment: 0.94.15 compiled with jdk6 running on jdk7 (but I don't
think it's relevant)
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Minor
Fix For: 0.94.25
I had TestDistributedLogSplitting.testRecoveredEdits fail on a internal test
rig. It's very infrequent.
The proximal failure looks like HBASE-9002, the assertion finds 0 edits
instead of 1000.
Looking through the logs, the #regions printed is 1
{code}
line 236LOG.info(#regions = + regions.size());
{code}
And then in the makeHLog method I never see any log entry for the actual
writing
{code}
line 569for (int i = 0; i n; i++) {
line 570 LOG.info(region + hris.get(i).getRegionNameAsString() +
has + counts[i] + edits);
line 571}
{code}
I think what's happening is we're picking a region that is META or ROOT, so
the checks after the #regions= line removes the region and we end up
inserting no data.
It looks like the patch for HBASE-9002 should fix this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12235) Backport to 0.94: HBASE-9002 TestDistributedLogSplitting.testRecoverdEdits should test correct RS
Sean Busbey created HBASE-12235:
---
Summary: Backport to 0.94: HBASE-9002
TestDistributedLogSplitting.testRecoverdEdits should test correct RS
Key: HBASE-12235
URL: https://issues.apache.org/jira/browse/HBASE-12235
Project: HBase
Issue Type: Task
Components: test
Affects Versions: 0.94.0
Environment: 0.94.15 compiled with jdk6 running on jdk7 (but I don't
think it's relevant)
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Minor
Fix For: 0.94.25
I had TestDistributedLogSplitting.testRecoveredEdits fail on a internal test
rig. It's very infrequent.
The proximal failure looks like HBASE-9002, the assertion finds 0 edits instead
of 1000.
Looking through the logs, the #regions printed is 0
{code}
line 236LOG.info(#regions = + regions.size());
{code}
And then in the makeHLog method I never see any log entry for the actual writing
{code}
line 569for (int i = 0; i n; i++) {
line 570 LOG.info(region + hris.get(i).getRegionNameAsString() + has
+ counts[i] + edits);
line 571}
{code}
I think what's happening is we're picking a region that is META or ROOT, so the
checks after the #regions= line removes the region and we end up inserting no
data.
It looks like the patch for HBASE-9002 should fix this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12235) Backport to 0.94: HBASE-9002 TestDistributedLogSplitting.testRecoverdEdits should test correct region
[
https://issues.apache.org/jira/browse/HBASE-12235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-12235:
Summary: Backport to 0.94: HBASE-9002
TestDistributedLogSplitting.testRecoverdEdits should test correct region (was:
Backport to 0.94: HBASE-9002 TestDistributedLogSplitting.testRecoverdEdits
should test correct RS)
Backport to 0.94: HBASE-9002 TestDistributedLogSplitting.testRecoverdEdits
should test correct region
-
Key: HBASE-12235
URL: https://issues.apache.org/jira/browse/HBASE-12235
Project: HBase
Issue Type: Task
Components: test
Affects Versions: 0.94.0
Environment: 0.94.15 compiled with jdk6 running on jdk7 (but I don't
think it's relevant)
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Minor
Fix For: 0.94.25
I had TestDistributedLogSplitting.testRecoveredEdits fail on a internal test
rig. It's very infrequent.
The proximal failure looks like HBASE-9002, the assertion finds 0 edits
instead of 1000.
Looking through the logs, the #regions printed is 0
{code}
line 236LOG.info(#regions = + regions.size());
{code}
And then in the makeHLog method I never see any log entry for the actual
writing
{code}
line 569for (int i = 0; i n; i++) {
line 570 LOG.info(region + hris.get(i).getRegionNameAsString() +
has + counts[i] + edits);
line 571}
{code}
I think what's happening is we're picking a region that is META or ROOT, so
the checks after the #regions= line removes the region and we end up
inserting no data.
It looks like the patch for HBASE-9002 should fix this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12235) Backport to 0.94: HBASE-9002 TestDistributedLogSplitting.testRecoverdEdits should test correct region
[
https://issues.apache.org/jira/browse/HBASE-12235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-12235:
Status: Patch Available (was: Open)
Backport to 0.94: HBASE-9002 TestDistributedLogSplitting.testRecoverdEdits
should test correct region
-
Key: HBASE-12235
URL: https://issues.apache.org/jira/browse/HBASE-12235
Project: HBase
Issue Type: Task
Components: test
Affects Versions: 0.94.0
Environment: 0.94.15 compiled with jdk6 running on jdk7 (but I don't
think it's relevant)
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Minor
Fix For: 0.94.25
Attachments: HBASE-12235-0.94.1.patch.txt
I had TestDistributedLogSplitting.testRecoveredEdits fail on a internal test
rig. It's very infrequent.
The proximal failure looks like HBASE-9002, the assertion finds 0 edits
instead of 1000.
Looking through the logs, the #regions printed is 1
{code}
line 236LOG.info(#regions = + regions.size());
{code}
And then in the makeHLog method I never see any log entry for the actual
writing
{code}
line 569for (int i = 0; i n; i++) {
line 570 LOG.info(region + hris.get(i).getRegionNameAsString() +
has + counts[i] + edits);
line 571}
{code}
I think what's happening is we're picking a region that is META or ROOT, so
the checks after the #regions= line removes the region and we end up
inserting no data.
It looks like the patch for HBASE-9002 should fix this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12235) Backport to 0.94: HBASE-9002 TestDistributedLogSplitting.testRecoverdEdits should test correct region
[
https://issues.apache.org/jira/browse/HBASE-12235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-12235:
Attachment: HBASE-12235-0.94.1.patch.txt
patch from HBASE-9002 rebased to work on 0.94.
ran {{mvn -Dtest=TestDistributedLogSplitting#testRecoveredEdits test}}
successfully.
Backport to 0.94: HBASE-9002 TestDistributedLogSplitting.testRecoverdEdits
should test correct region
-
Key: HBASE-12235
URL: https://issues.apache.org/jira/browse/HBASE-12235
Project: HBase
Issue Type: Task
Components: test
Affects Versions: 0.94.0
Environment: 0.94.15 compiled with jdk6 running on jdk7 (but I don't
think it's relevant)
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Minor
Fix For: 0.94.25
Attachments: HBASE-12235-0.94.1.patch.txt
I had TestDistributedLogSplitting.testRecoveredEdits fail on a internal test
rig. It's very infrequent.
The proximal failure looks like HBASE-9002, the assertion finds 0 edits
instead of 1000.
Looking through the logs, the #regions printed is 1
{code}
line 236LOG.info(#regions = + regions.size());
{code}
And then in the makeHLog method I never see any log entry for the actual
writing
{code}
line 569for (int i = 0; i n; i++) {
line 570 LOG.info(region + hris.get(i).getRegionNameAsString() +
has + counts[i] + edits);
line 571}
{code}
I think what's happening is we're picking a region that is META or ROOT, so
the checks after the #regions= line removes the region and we end up
inserting no data.
It looks like the patch for HBASE-9002 should fix this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12235) Backport to 0.94: HBASE-9002 TestDistributedLogSplitting.testRecoverdEdits should test correct region
[
https://issues.apache.org/jira/browse/HBASE-12235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168055#comment-14168055
]
Hadoop QA commented on HBASE-12235:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12674347/HBASE-12235-0.94.1.patch.txt
against trunk revision .
ATTACHMENT ID: 12674347
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11313//console
This message is automatically generated.
Backport to 0.94: HBASE-9002 TestDistributedLogSplitting.testRecoverdEdits
should test correct region
-
Key: HBASE-12235
URL: https://issues.apache.org/jira/browse/HBASE-12235
Project: HBase
Issue Type: Task
Components: test
Affects Versions: 0.94.0
Environment: 0.94.15 compiled with jdk6 running on jdk7 (but I don't
think it's relevant)
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Minor
Fix For: 0.94.25
Attachments: HBASE-12235-0.94.1.patch.txt
I had TestDistributedLogSplitting.testRecoveredEdits fail on a internal test
rig. It's very infrequent.
The proximal failure looks like HBASE-9002, the assertion finds 0 edits
instead of 1000.
Looking through the logs, the #regions printed is 1
{code}
line 236LOG.info(#regions = + regions.size());
{code}
And then in the makeHLog method I never see any log entry for the actual
writing
{code}
line 569for (int i = 0; i n; i++) {
line 570 LOG.info(region + hris.get(i).getRegionNameAsString() +
has + counts[i] + edits);
line 571}
{code}
I think what's happening is we're picking a region that is META or ROOT, so
the checks after the #regions= line removes the region and we end up
inserting no data.
It looks like the patch for HBASE-9002 should fix this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11890) HBase REST Client is hard coded to http protocol
[ https://issues.apache.org/jira/browse/HBASE-11890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168065#comment-14168065 ] Qiang Tian commented on HBASE-11890: Hi [~stack], there is also a doc update :-) HBase-11890-doc.patch HBase REST Client is hard coded to http protocol Key: HBASE-11890 URL: https://issues.apache.org/jira/browse/HBASE-11890 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.96.2 Reporter: Eric Yang Assignee: Qiang Tian Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBase-11890-doc.patch, HBase-11890-master-v1.patch, HBase-11890-master.patch HBase REST Client executePathOnly only supports http. It would be nice if there is a option to enable REST API client to connect through SSL. org.apache.hadoop.hbase.rest.client.Cluster class does not indicate which protocol can be used, we can either set flag in Cluster class or introduce a parameter in Client class to toggle SSL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12229) NullPointerException in SnapshotTestingUtils
[
https://issues.apache.org/jira/browse/HBASE-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168219#comment-14168219
]
Andrew Purtell commented on HBASE-12229:
This begs the question why assignment isn't a necessary precondition for a
table to be online (it is of course):
{code}
@@ -643,6 +643,7 @@ public class SnapshotTestingUtils {
}
byte[][] splitKeys = getSplitKeys();
util.getHBaseAdmin().createTable(htd, splitKeys);
+util.waitUntilAllRegionsAssigned(tableName);
waitForTableToBeOnline(util, tableName);
assertEquals((splitKeys.length + 1) * regionReplication,
util.getHBaseAdmin().getTableRegions(tableName).size());
{code}
bq. In short, some tests in TestRestoreSnapshot... create a table and then
invoke SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes
that regions have been assigned by the time it's invoked
Why not fix that?
NullPointerException in SnapshotTestingUtils
Key: HBASE-12229
URL: https://issues.apache.org/jira/browse/HBASE-12229
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 1.0.0, 2.0.0, 0.98.7
Reporter: Dima Spivak
Assignee: Dima Spivak
Priority: Minor
Attachments: HBASE-12229.patch, HBASE-12229_v1.patch
I tracked down occasional flakiness in TestRestoreSnapshotFromClient to a
potential NPE in SnapshotTestingUtils#waitForTableToBeOnline. In short, some
tests in TestRestoreSnapshot... create a table and then invoke
SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes that
regions have been assigned by the time it's invoked (which is not always the
case).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12233) Bring back root table
[
https://issues.apache.org/jira/browse/HBASE-12233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168226#comment-14168226
]
Andrew Purtell commented on HBASE-12233:
{quote}
I am for sure against bringing root back in for branch-1
bq. For sure root will not be back for branch-1.
{quote}
That does make this a thought exercise then, since there's no credible near
term deployment for code checked only into master. My understanding is [~virag]
and crew are actually looking for split meta to become available as an
off-by-default option in 0.98. If there are strong feelings about bringing back
root for 0.98 and branch-1 let's veto this and move on. Deploys that would like
to try it can maintain custom patches. Or, if that's an undesirable outcome,
let's not for sure be against anything so soon. If we're getting patches that
implement this feature, let's put them in a feature branch and compare and
contrast in the deployment where there's a problem to solve. With data in hand
and operational feedback about change stability the discussion would be more
grounded than current.
Bring back root table
-
Key: HBASE-12233
URL: https://issues.apache.org/jira/browse/HBASE-12233
Project: HBase
Issue Type: Sub-task
Reporter: Virag Kothari
Assignee: Virag Kothari
Attachments: HBASE-12233.patch
First step towards splitting meta.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12233) Bring back root table
[ https://issues.apache.org/jira/browse/HBASE-12233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168227#comment-14168227 ] Andrew Purtell commented on HBASE-12233: Adding to the above. If bringing back root and split meta are off-by-default options, with a rolling upgrade story, in theory it can (and, IMO should) not make releases 1.0.x but could make it into release 1.1.0. That's different from vetoing a change from branch-1 entirely. Bring back root table - Key: HBASE-12233 URL: https://issues.apache.org/jira/browse/HBASE-12233 Project: HBase Issue Type: Sub-task Reporter: Virag Kothari Assignee: Virag Kothari Attachments: HBASE-12233.patch First step towards splitting meta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11542) Unit Test KeyStoreTestUtil.java compilation failure in IBM JDK
[ https://issues.apache.org/jira/browse/HBASE-11542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11542: --- Status: Open (was: Patch Available) Unit Test KeyStoreTestUtil.java compilation failure in IBM JDK Key: HBASE-11542 URL: https://issues.apache.org/jira/browse/HBASE-11542 Project: HBase Issue Type: Improvement Components: build, test Affects Versions: 0.99.0 Environment: RHEL 6.3 ,IBM JDK 6 Reporter: LinseyPang Priority: Minor Fix For: 2.0.0 Attachments: HBASE-11542-4.patch, HBASE-11542-5.patch, HBASE_11542-1.patch, hbase11542-0.99-v3.patch, hbase11542-0.99-v3.patch, hbase11542-0.99-v3.patch, hbase_11542-v2.patch, sslkeystore.patch In trunk, jira HBase-10336 added a utility test KeyStoreTestUtil.java, which leverages the following sun classes: import sun.security.x509.AlgorithmId; import sun.security.x509.CertificateAlgorithmId; this cause hbase compiler failure if using IBM JDK, There are similar classes like below in IBM jdk: import com.ibm.security.x509.AlgorithmId; import com.ibm.security.x509.CertificateAlgorithmId; This jira is to add handling of the x509 references. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12006) [JDK 8] KeyStoreTestUtil#generateCertificate fails due to subject class type invalid
[
https://issues.apache.org/jira/browse/HBASE-12006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168246#comment-14168246
]
Andrew Purtell commented on HBASE-12006:
This could be addressed by HBASE-11542. Instead of using internal security APIs
which are 1) nonstandard and 2) at least in the case of Oracle Java going to
disappear, we can instead use keytool once to generate a test file, stringify
it, and use this in the test.
[JDK 8] KeyStoreTestUtil#generateCertificate fails due to subject class type
invalid
--
Key: HBASE-12006
URL: https://issues.apache.org/jira/browse/HBASE-12006
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0, 2.0.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Minor
Running tests on Java 8. All unit tests for branch 0.98 pass. On master
branch some variation in the security API is causing a failure in
TestSSLHttpServer:
{noformat}
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.181 sec
FAILURE! - in org.apache.hadoop.hbase.http.TestSSLHttpServer
org.apache.hadoop.hbase.http.TestSSLHttpServer Time elapsed: 0.181 sec
ERROR!
java.security.cert.CertificateException: Subject class type invalid.
at sun.security.x509.X509CertInfo.setSubject(X509CertInfo.java:888)
at sun.security.x509.X509CertInfo.set(X509CertInfo.java:415)
at
org.apache.hadoop.hbase.http.ssl.KeyStoreTestUtil.generateCertificate(KeyStoreTestUtil.java:94)
at
org.apache.hadoop.hbase.http.ssl.KeyStoreTestUtil.setupSSLConfig(KeyStoreTestUtil.java:246)
at
org.apache.hadoop.hbase.http.TestSSLHttpServer.setup(TestSSLHttpServer.java:72)
org.apache.hadoop.hbase.http.TestSSLHttpServer Time elapsed: 0.181 sec
ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.hbase.http.TestSSLHttpServer.cleanup(TestSSLHttpServer.java:100)
Tests in error:
TestSSLHttpServer.setup:72 » Certificate Subject class type invalid.
TestSSLHttpServer.cleanup:100 NullPointer
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12006) [JDK 8] KeyStoreTestUtil#generateCertificate fails due to subject class type invalid
[
https://issues.apache.org/jira/browse/HBASE-12006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-12006:
---
Assignee: (was: Andrew Purtell)
[JDK 8] KeyStoreTestUtil#generateCertificate fails due to subject class type
invalid
--
Key: HBASE-12006
URL: https://issues.apache.org/jira/browse/HBASE-12006
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0, 2.0.0
Reporter: Andrew Purtell
Priority: Minor
Running tests on Java 8. All unit tests for branch 0.98 pass. On master
branch some variation in the security API is causing a failure in
TestSSLHttpServer:
{noformat}
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.181 sec
FAILURE! - in org.apache.hadoop.hbase.http.TestSSLHttpServer
org.apache.hadoop.hbase.http.TestSSLHttpServer Time elapsed: 0.181 sec
ERROR!
java.security.cert.CertificateException: Subject class type invalid.
at sun.security.x509.X509CertInfo.setSubject(X509CertInfo.java:888)
at sun.security.x509.X509CertInfo.set(X509CertInfo.java:415)
at
org.apache.hadoop.hbase.http.ssl.KeyStoreTestUtil.generateCertificate(KeyStoreTestUtil.java:94)
at
org.apache.hadoop.hbase.http.ssl.KeyStoreTestUtil.setupSSLConfig(KeyStoreTestUtil.java:246)
at
org.apache.hadoop.hbase.http.TestSSLHttpServer.setup(TestSSLHttpServer.java:72)
org.apache.hadoop.hbase.http.TestSSLHttpServer Time elapsed: 0.181 sec
ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.hbase.http.TestSSLHttpServer.cleanup(TestSSLHttpServer.java:100)
Tests in error:
TestSSLHttpServer.setup:72 » Certificate Subject class type invalid.
TestSSLHttpServer.cleanup:100 NullPointer
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12228) Backport HBASE-11373 (hbase-protocol compile failed for name conflict of RegionTransition) to 0.98
[
https://issues.apache.org/jira/browse/HBASE-12228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168251#comment-14168251
]
Andrew Purtell commented on HBASE-12228:
Any chance of a quick review [~jxiang]? This is a straightforward backport of
your original change.
Backport HBASE-11373 (hbase-protocol compile failed for name conflict of
RegionTransition) to 0.98
--
Key: HBASE-12228
URL: https://issues.apache.org/jira/browse/HBASE-12228
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 0.98.8
Attachments: HBASE-12228-0.98.patch
{quote}
RegionServerStatus.proto:81:9: RegionTransition is already defined in file
ZooKeeper.proto.
RegionServerStatus.proto:114:12: RegionTransition seems to be defined in
ZooKeeper.proto, which is not imported by RegionServerStatus.proto. To
use it here, please add the necessary import.
{quote}
This was introduced into 0.98 in e6ffa86e
{noformat}
commit e6ffa86e33ee173afcff15ca4b614e6ec56357ed
Author: Andrew Purtell apurt...@apache.org
Date: Tue Aug 26 08:01:09 2014 -0700
HBASE-11546 Backport ZK-less region assignment to 0.98 (Virag Kothari)
[1/8]
HBASE-11059 ZK-less region assignment (Jimmy Xiang
{noformat}
There's a later fix for this that needs to be applied:
{noformat}
commit 175f133dbc127d7eb2ba5693cc6b2e4fe3c51655
Author: Jimmy Xiang jxi...@cloudera.com
Date: Wed Jun 18 08:38:05 2014 -0700
HBASE-11373 hbase-protocol compile failed for name conflict of
RegionTransition
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12228) Backport HBASE-11373 (hbase-protocol compile failed for name conflict of RegionTransition) to 0.98
[
https://issues.apache.org/jira/browse/HBASE-12228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168264#comment-14168264
]
Jimmy Xiang commented on HBASE-12228:
-
+1. Thanks!
Backport HBASE-11373 (hbase-protocol compile failed for name conflict of
RegionTransition) to 0.98
--
Key: HBASE-12228
URL: https://issues.apache.org/jira/browse/HBASE-12228
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 0.98.8
Attachments: HBASE-12228-0.98.patch
{quote}
RegionServerStatus.proto:81:9: RegionTransition is already defined in file
ZooKeeper.proto.
RegionServerStatus.proto:114:12: RegionTransition seems to be defined in
ZooKeeper.proto, which is not imported by RegionServerStatus.proto. To
use it here, please add the necessary import.
{quote}
This was introduced into 0.98 in e6ffa86e
{noformat}
commit e6ffa86e33ee173afcff15ca4b614e6ec56357ed
Author: Andrew Purtell apurt...@apache.org
Date: Tue Aug 26 08:01:09 2014 -0700
HBASE-11546 Backport ZK-less region assignment to 0.98 (Virag Kothari)
[1/8]
HBASE-11059 ZK-less region assignment (Jimmy Xiang
{noformat}
There's a later fix for this that needs to be applied:
{noformat}
commit 175f133dbc127d7eb2ba5693cc6b2e4fe3c51655
Author: Jimmy Xiang jxi...@cloudera.com
Date: Wed Jun 18 08:38:05 2014 -0700
HBASE-11373 hbase-protocol compile failed for name conflict of
RegionTransition
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12235) Backport to 0.94: HBASE-9002 TestDistributedLogSplitting.testRecoverdEdits should test correct region
[
https://issues.apache.org/jira/browse/HBASE-12235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168268#comment-14168268
]
Lars Hofhansl commented on HBASE-12235:
---
Looks good. +1
Backport to 0.94: HBASE-9002 TestDistributedLogSplitting.testRecoverdEdits
should test correct region
-
Key: HBASE-12235
URL: https://issues.apache.org/jira/browse/HBASE-12235
Project: HBase
Issue Type: Task
Components: test
Affects Versions: 0.94.0
Environment: 0.94.15 compiled with jdk6 running on jdk7 (but I don't
think it's relevant)
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Minor
Fix For: 0.94.25
Attachments: HBASE-12235-0.94.1.patch.txt
I had TestDistributedLogSplitting.testRecoveredEdits fail on a internal test
rig. It's very infrequent.
The proximal failure looks like HBASE-9002, the assertion finds 0 edits
instead of 1000.
Looking through the logs, the #regions printed is 1
{code}
line 236LOG.info(#regions = + regions.size());
{code}
And then in the makeHLog method I never see any log entry for the actual
writing
{code}
line 569for (int i = 0; i n; i++) {
line 570 LOG.info(region + hris.get(i).getRegionNameAsString() +
has + counts[i] + edits);
line 571}
{code}
I think what's happening is we're picking a region that is META or ROOT, so
the checks after the #regions= line removes the region and we end up
inserting no data.
It looks like the patch for HBASE-9002 should fix this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12228) Backport HBASE-11373 (hbase-protocol compile failed for name conflict of RegionTransition) to 0.98
[
https://issues.apache.org/jira/browse/HBASE-12228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-12228:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Pushed to 0.98. Thanks for the review [~jxiang]
Backport HBASE-11373 (hbase-protocol compile failed for name conflict of
RegionTransition) to 0.98
--
Key: HBASE-12228
URL: https://issues.apache.org/jira/browse/HBASE-12228
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 0.98.8
Attachments: HBASE-12228-0.98.patch
{quote}
RegionServerStatus.proto:81:9: RegionTransition is already defined in file
ZooKeeper.proto.
RegionServerStatus.proto:114:12: RegionTransition seems to be defined in
ZooKeeper.proto, which is not imported by RegionServerStatus.proto. To
use it here, please add the necessary import.
{quote}
This was introduced into 0.98 in e6ffa86e
{noformat}
commit e6ffa86e33ee173afcff15ca4b614e6ec56357ed
Author: Andrew Purtell apurt...@apache.org
Date: Tue Aug 26 08:01:09 2014 -0700
HBASE-11546 Backport ZK-less region assignment to 0.98 (Virag Kothari)
[1/8]
HBASE-11059 ZK-less region assignment (Jimmy Xiang
{noformat}
There's a later fix for this that needs to be applied:
{noformat}
commit 175f133dbc127d7eb2ba5693cc6b2e4fe3c51655
Author: Jimmy Xiang jxi...@cloudera.com
Date: Wed Jun 18 08:38:05 2014 -0700
HBASE-11373 hbase-protocol compile failed for name conflict of
RegionTransition
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12233) Bring back root table
[ https://issues.apache.org/jira/browse/HBASE-12233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168325#comment-14168325 ] Virag Kothari commented on HBASE-12233: --- bq. actually looking for split meta to become available as an off-by-default option in 0.98 Yes, we are looking for root and split meta as an off-by-default option in 0.98. Stack had a suggestion of doing this hiding of root by adding an pbuf entry in meta znode which determines whether it holds the meta location or root location. Depending on this field, the clients will know the presence of root. Master will have a config depending on which it will create root table and set this field in meta znode. For 0.98/branch-1, we can have this config in master to be turned off by default so no root stuff is touched and the clients wont know the presence of root at all. Also, clients wont need upgrade when the master turns on the switch for root. Bring back root table - Key: HBASE-12233 URL: https://issues.apache.org/jira/browse/HBASE-12233 Project: HBase Issue Type: Sub-task Reporter: Virag Kothari Assignee: Virag Kothari Attachments: HBASE-12233.patch First step towards splitting meta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12233) Bring back root table
[ https://issues.apache.org/jira/browse/HBASE-12233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168328#comment-14168328 ] Virag Kothari commented on HBASE-12233: --- bq. Right now meta gets 2x more reads than writes on real production clusters Write throughput on meta become important during cluster startups, operations like create/enable/disable and during rack failures. Any assignments involved during such activity usually does minimum 2 writes to meta for a single region as each region goes through at least 2 states. So even in best case scenarios, there will be 2M writes just for bringing a table with 1M regions online. Although, a split meta would help with both reads and writes as discussed over in parent JIRA. Bring back root table - Key: HBASE-12233 URL: https://issues.apache.org/jira/browse/HBASE-12233 Project: HBase Issue Type: Sub-task Reporter: Virag Kothari Assignee: Virag Kothari Attachments: HBASE-12233.patch First step towards splitting meta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12229) NullPointerException in SnapshotTestingUtils
[ https://issues.apache.org/jira/browse/HBASE-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168329#comment-14168329 ] Dima Spivak commented on HBASE-12229: - That's a fair point, [~apurtell], and I'll post up a patch doing that. Is it worth separately addressing why the [synchronous] HBaseAdmin#createTable method returns before a table is actually ready to accept puts? If that was fixed, there wouldn't need to be these helper methods checking for regions being available in HBTU. NullPointerException in SnapshotTestingUtils Key: HBASE-12229 URL: https://issues.apache.org/jira/browse/HBASE-12229 Project: HBase Issue Type: Bug Components: test Affects Versions: 1.0.0, 2.0.0, 0.98.7 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Minor Attachments: HBASE-12229.patch, HBASE-12229_v1.patch I tracked down occasional flakiness in TestRestoreSnapshotFromClient to a potential NPE in SnapshotTestingUtils#waitForTableToBeOnline. In short, some tests in TestRestoreSnapshot... create a table and then invoke SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes that regions have been assigned by the time it's invoked (which is not always the case). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11373) hbase-protocol compile failed for name conflict of RegionTransition
[
https://issues.apache.org/jira/browse/HBASE-11373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168338#comment-14168338
]
Hudson commented on HBASE-11373:
FAILURE: Integrated in HBase-0.98 #593 (See
[https://builds.apache.org/job/HBase-0.98/593/])
HBASE-12228 Backport HBASE-11373 (hbase-protocol compile failed for name
conflict of RegionTransition) to 0.98 (apurtell: rev
f5f81cc791c7770b3e797c33eedc93bdaac211e7)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransaction.java
*
hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionServerStatusProtos.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MapReduceProtos.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
* hbase-protocol/src/main/protobuf/RegionServerStatus.proto
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/CloseRegionHandler.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/MockRegionServerServices.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
hbase-protocol compile failed for name conflict of RegionTransition
---
Key: HBASE-11373
URL: https://issues.apache.org/jira/browse/HBASE-11373
Project: HBase
Issue Type: Bug
Components: Protobufs
Reporter: Liu Shaohui
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.99.0
Attachments: hbase-11373.patch
The compile of hbase-protocol failed for there are two message named
RegionTransition in ZooKeeper.proto and RegionServerStatus.proto
{quote}
$mvn clean package -Pcompile-protobuf -X
\[DEBUG\] RegionServerStatus.proto:81:9: RegionTransition is already
defined in file ZooKeeper.proto.
\[DEBUG\] RegionServerStatus.proto:114:12: RegionTransition seems to be
defined in ZooKeeper.proto, which is not imported by
RegionServerStatus.proto. To use it here, please add the necessary import.
\[ERROR\] protoc compiler error
{quote}
Through It will be ok if we compile the the ZooKeeper.proto and
RegionServerStatus.proto seperately. it is not very convenient.
The new RegionTransition is RegionServerStatus.proto and introduced in
HBASE-11059.
[~jxiang]
What's your suggestion about this issue?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12228) Backport HBASE-11373 (hbase-protocol compile failed for name conflict of RegionTransition) to 0.98
[
https://issues.apache.org/jira/browse/HBASE-12228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168337#comment-14168337
]
Hudson commented on HBASE-12228:
FAILURE: Integrated in HBase-0.98 #593 (See
[https://builds.apache.org/job/HBase-0.98/593/])
HBASE-12228 Backport HBASE-11373 (hbase-protocol compile failed for name
conflict of RegionTransition) to 0.98 (apurtell: rev
f5f81cc791c7770b3e797c33eedc93bdaac211e7)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransaction.java
*
hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionServerStatusProtos.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MapReduceProtos.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
* hbase-protocol/src/main/protobuf/RegionServerStatus.proto
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/CloseRegionHandler.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/MockRegionServerServices.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
Backport HBASE-11373 (hbase-protocol compile failed for name conflict of
RegionTransition) to 0.98
--
Key: HBASE-12228
URL: https://issues.apache.org/jira/browse/HBASE-12228
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 0.98.8
Attachments: HBASE-12228-0.98.patch
{quote}
RegionServerStatus.proto:81:9: RegionTransition is already defined in file
ZooKeeper.proto.
RegionServerStatus.proto:114:12: RegionTransition seems to be defined in
ZooKeeper.proto, which is not imported by RegionServerStatus.proto. To
use it here, please add the necessary import.
{quote}
This was introduced into 0.98 in e6ffa86e
{noformat}
commit e6ffa86e33ee173afcff15ca4b614e6ec56357ed
Author: Andrew Purtell apurt...@apache.org
Date: Tue Aug 26 08:01:09 2014 -0700
HBASE-11546 Backport ZK-less region assignment to 0.98 (Virag Kothari)
[1/8]
HBASE-11059 ZK-less region assignment (Jimmy Xiang
{noformat}
There's a later fix for this that needs to be applied:
{noformat}
commit 175f133dbc127d7eb2ba5693cc6b2e4fe3c51655
Author: Jimmy Xiang jxi...@cloudera.com
Date: Wed Jun 18 08:38:05 2014 -0700
HBASE-11373 hbase-protocol compile failed for name conflict of
RegionTransition
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11373) hbase-protocol compile failed for name conflict of RegionTransition
[
https://issues.apache.org/jira/browse/HBASE-11373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168358#comment-14168358
]
Hudson commented on HBASE-11373:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #564 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/564/])
HBASE-12228 Backport HBASE-11373 (hbase-protocol compile failed for name
conflict of RegionTransition) to 0.98 (apurtell: rev
f5f81cc791c7770b3e797c33eedc93bdaac211e7)
*
hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionServerStatusProtos.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/CloseRegionHandler.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/MockRegionServerServices.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransaction.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* hbase-protocol/src/main/protobuf/RegionServerStatus.proto
*
hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MapReduceProtos.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
hbase-protocol compile failed for name conflict of RegionTransition
---
Key: HBASE-11373
URL: https://issues.apache.org/jira/browse/HBASE-11373
Project: HBase
Issue Type: Bug
Components: Protobufs
Reporter: Liu Shaohui
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.99.0
Attachments: hbase-11373.patch
The compile of hbase-protocol failed for there are two message named
RegionTransition in ZooKeeper.proto and RegionServerStatus.proto
{quote}
$mvn clean package -Pcompile-protobuf -X
\[DEBUG\] RegionServerStatus.proto:81:9: RegionTransition is already
defined in file ZooKeeper.proto.
\[DEBUG\] RegionServerStatus.proto:114:12: RegionTransition seems to be
defined in ZooKeeper.proto, which is not imported by
RegionServerStatus.proto. To use it here, please add the necessary import.
\[ERROR\] protoc compiler error
{quote}
Through It will be ok if we compile the the ZooKeeper.proto and
RegionServerStatus.proto seperately. it is not very convenient.
The new RegionTransition is RegionServerStatus.proto and introduced in
HBASE-11059.
[~jxiang]
What's your suggestion about this issue?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12228) Backport HBASE-11373 (hbase-protocol compile failed for name conflict of RegionTransition) to 0.98
[
https://issues.apache.org/jira/browse/HBASE-12228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168357#comment-14168357
]
Hudson commented on HBASE-12228:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #564 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/564/])
HBASE-12228 Backport HBASE-11373 (hbase-protocol compile failed for name
conflict of RegionTransition) to 0.98 (apurtell: rev
f5f81cc791c7770b3e797c33eedc93bdaac211e7)
*
hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionServerStatusProtos.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/CloseRegionHandler.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/MockRegionServerServices.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransaction.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* hbase-protocol/src/main/protobuf/RegionServerStatus.proto
*
hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MapReduceProtos.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
Backport HBASE-11373 (hbase-protocol compile failed for name conflict of
RegionTransition) to 0.98
--
Key: HBASE-12228
URL: https://issues.apache.org/jira/browse/HBASE-12228
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 0.98.8
Attachments: HBASE-12228-0.98.patch
{quote}
RegionServerStatus.proto:81:9: RegionTransition is already defined in file
ZooKeeper.proto.
RegionServerStatus.proto:114:12: RegionTransition seems to be defined in
ZooKeeper.proto, which is not imported by RegionServerStatus.proto. To
use it here, please add the necessary import.
{quote}
This was introduced into 0.98 in e6ffa86e
{noformat}
commit e6ffa86e33ee173afcff15ca4b614e6ec56357ed
Author: Andrew Purtell apurt...@apache.org
Date: Tue Aug 26 08:01:09 2014 -0700
HBASE-11546 Backport ZK-less region assignment to 0.98 (Virag Kothari)
[1/8]
HBASE-11059 ZK-less region assignment (Jimmy Xiang
{noformat}
There's a later fix for this that needs to be applied:
{noformat}
commit 175f133dbc127d7eb2ba5693cc6b2e4fe3c51655
Author: Jimmy Xiang jxi...@cloudera.com
Date: Wed Jun 18 08:38:05 2014 -0700
HBASE-11373 hbase-protocol compile failed for name conflict of
RegionTransition
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-12072) We are doing 35 x 35 retries for master operations
[
https://issues.apache.org/jira/browse/HBASE-12072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar reassigned HBASE-12072:
-
Assignee: Enis Soztutar
We are doing 35 x 35 retries for master operations
--
Key: HBASE-12072
URL: https://issues.apache.org/jira/browse/HBASE-12072
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.6
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Attachments: 12072-v1.txt, 12072-v2.txt
For master requests, there are two retry mechanisms in effect. The first one
is from HBaseAdmin.executeCallable()
{code}
private V V executeCallable(MasterCallableV callable) throws
IOException {
RpcRetryingCallerV caller = rpcCallerFactory.newCaller();
try {
return caller.callWithRetries(callable);
} finally {
callable.close();
}
}
{code}
And inside, the other one is from StubMaker.makeStub():
{code}
/**
* Create a stub against the master. Retry if necessary.
* @return A stub to do codeintf/code against the master
* @throws MasterNotRunningException
*/
@edu.umd.cs.findbugs.annotations.SuppressWarnings
(value=SWL_SLEEP_WITH_LOCK_HELD)
Object makeStub() throws MasterNotRunningException {
{code}
The tests will just hang for 10 min * 35 ~= 6hours.
{code}
2014-09-23 16:19:05,151 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 1 of 35
failed; retrying after sleep of 100, exception=java.io.IOException: Can't get
master address from ZooKeeper; znode data == null
2014-09-23 16:19:05,253 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 2 of 35
failed; retrying after sleep of 200, exception=java.io.IOException: Can't get
master address from ZooKeeper; znode data == null
2014-09-23 16:19:05,456 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 3 of 35
failed; retrying after sleep of 300, exception=java.io.IOException: Can't get
master address from ZooKeeper; znode data == null
2014-09-23 16:19:05,759 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 4 of 35
failed; retrying after sleep of 500, exception=java.io.IOException: Can't get
master address from ZooKeeper; znode data == null
2014-09-23 16:19:06,262 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 5 of 35
failed; retrying after sleep of 1008, exception=java.io.IOException: Can't
get master address from ZooKeeper; znode data == null
2014-09-23 16:19:07,273 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 6 of 35
failed; retrying after sleep of 2011, exception=java.io.IOException: Can't
get master address from ZooKeeper; znode data == null
2014-09-23 16:19:09,286 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 7 of 35
failed; retrying after sleep of 4012, exception=java.io.IOException: Can't
get master address from ZooKeeper; znode data == null
2014-09-23 16:19:13,303 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 8 of 35
failed; retrying after sleep of 10033, exception=java.io.IOException: Can't
get master address from ZooKeeper; znode data == null
2014-09-23 16:19:23,343 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 9 of 35
failed; retrying after sleep of 10089, exception=java.io.IOException: Can't
get master address from ZooKeeper; znode data == null
2014-09-23 16:19:33,439 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 10 of
35 failed; retrying after sleep of 10027, exception=java.io.IOException:
Can't get master address from ZooKeeper; znode data == null
2014-09-23 16:19:43,473 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 11 of
35 failed; retrying after sleep of 10004, exception=java.io.IOException:
Can't get master address from ZooKeeper; znode data == null
2014-09-23 16:19:53,485 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 12 of
35 failed; retrying after sleep of 20160, exception=java.io.IOException:
Can't get master address from ZooKeeper; znode data == null
2014-09-23 16:20:13,656 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 13 of
35 failed; retrying after sleep of 20006, exception=java.io.IOException:
Can't get master address from ZooKeeper; znode data == null
2014-09-23 16:20:33,675 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 14 of
35 failed; retrying after sleep of 20076, exception=java.io.IOException:
Can't get master
[jira] [Updated] (HBASE-12072) We are doing 35 x 35 retries for master operations
[
https://issues.apache.org/jira/browse/HBASE-12072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-12072:
--
Attachment: hbase-12072_v1.patch
Here is my attempt for a patch. It removed retries in the makeStub in
ConnectionManager, but adds retries at the HBaseAdmin layer. We were not
retrying some operations before (move region etc), now they are changed to be
retryable as well.
Lets see what qa thinks.
We are doing 35 x 35 retries for master operations
--
Key: HBASE-12072
URL: https://issues.apache.org/jira/browse/HBASE-12072
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.6
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Attachments: 12072-v1.txt, 12072-v2.txt, hbase-12072_v1.patch
For master requests, there are two retry mechanisms in effect. The first one
is from HBaseAdmin.executeCallable()
{code}
private V V executeCallable(MasterCallableV callable) throws
IOException {
RpcRetryingCallerV caller = rpcCallerFactory.newCaller();
try {
return caller.callWithRetries(callable);
} finally {
callable.close();
}
}
{code}
And inside, the other one is from StubMaker.makeStub():
{code}
/**
* Create a stub against the master. Retry if necessary.
* @return A stub to do codeintf/code against the master
* @throws MasterNotRunningException
*/
@edu.umd.cs.findbugs.annotations.SuppressWarnings
(value=SWL_SLEEP_WITH_LOCK_HELD)
Object makeStub() throws MasterNotRunningException {
{code}
The tests will just hang for 10 min * 35 ~= 6hours.
{code}
2014-09-23 16:19:05,151 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 1 of 35
failed; retrying after sleep of 100, exception=java.io.IOException: Can't get
master address from ZooKeeper; znode data == null
2014-09-23 16:19:05,253 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 2 of 35
failed; retrying after sleep of 200, exception=java.io.IOException: Can't get
master address from ZooKeeper; znode data == null
2014-09-23 16:19:05,456 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 3 of 35
failed; retrying after sleep of 300, exception=java.io.IOException: Can't get
master address from ZooKeeper; znode data == null
2014-09-23 16:19:05,759 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 4 of 35
failed; retrying after sleep of 500, exception=java.io.IOException: Can't get
master address from ZooKeeper; znode data == null
2014-09-23 16:19:06,262 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 5 of 35
failed; retrying after sleep of 1008, exception=java.io.IOException: Can't
get master address from ZooKeeper; znode data == null
2014-09-23 16:19:07,273 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 6 of 35
failed; retrying after sleep of 2011, exception=java.io.IOException: Can't
get master address from ZooKeeper; znode data == null
2014-09-23 16:19:09,286 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 7 of 35
failed; retrying after sleep of 4012, exception=java.io.IOException: Can't
get master address from ZooKeeper; znode data == null
2014-09-23 16:19:13,303 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 8 of 35
failed; retrying after sleep of 10033, exception=java.io.IOException: Can't
get master address from ZooKeeper; znode data == null
2014-09-23 16:19:23,343 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 9 of 35
failed; retrying after sleep of 10089, exception=java.io.IOException: Can't
get master address from ZooKeeper; znode data == null
2014-09-23 16:19:33,439 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 10 of
35 failed; retrying after sleep of 10027, exception=java.io.IOException:
Can't get master address from ZooKeeper; znode data == null
2014-09-23 16:19:43,473 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 11 of
35 failed; retrying after sleep of 10004, exception=java.io.IOException:
Can't get master address from ZooKeeper; znode data == null
2014-09-23 16:19:53,485 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 12 of
35 failed; retrying after sleep of 20160, exception=java.io.IOException:
Can't get master address from ZooKeeper; znode data == null
2014-09-23 16:20:13,656 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 13 of
35 failed; retrying after sleep of 20006,
[jira] [Updated] (HBASE-11181) prune/delete old backups
[ https://issues.apache.org/jira/browse/HBASE-11181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11181: -- Fix Version/s: (was: 0.99.1) 0.99.2 prune/delete old backups Key: HBASE-11181 URL: https://issues.apache.org/jira/browse/HBASE-11181 Project: HBase Issue Type: New Feature Affects Versions: 0.99.0 Reporter: Jerry He Assignee: Enoch Hsu Fix For: 0.99.2 Attachments: HBASE-11181-V1.patch, HBASE-11181-contains-HBASE-11085-10900.patch Prune and delete old backups when they are no longer needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11148: -- Fix Version/s: (was: 0.99.1) 0.99.2 Provide a distributed procedure to globally roll logs - Key: HBASE-11148 URL: https://issues.apache.org/jira/browse/HBASE-11148 Project: HBase Issue Type: New Feature Reporter: Jerry He Assignee: Jerry He Fix For: 2.0.0, 0.99.2 Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch Propose a distributed procedure here to globally roll logs. Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a single RS. Some use cases may require that all the RS roll the logs at the same time and in a coordinated way. Also there may be requirement that some tasks be done together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168377#comment-14168377 ] Enis Soztutar commented on HBASE-11148: --- Ping [~jmhsieh]. This looks like almost there. Provide a distributed procedure to globally roll logs - Key: HBASE-11148 URL: https://issues.apache.org/jira/browse/HBASE-11148 Project: HBase Issue Type: New Feature Reporter: Jerry He Assignee: Jerry He Fix For: 2.0.0, 0.99.2 Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch Propose a distributed procedure here to globally roll logs. Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a single RS. Some use cases may require that all the RS roll the logs at the same time and in a coordinated way. Also there may be requirement that some tasks be done together with the roll log on each region server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11085) Incremental Backup Restore support
[
https://issues.apache.org/jira/browse/HBASE-11085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-11085:
--
Fix Version/s: (was: 0.99.1)
0.992
Incremental Backup Restore support
--
Key: HBASE-11085
URL: https://issues.apache.org/jira/browse/HBASE-11085
Project: HBase
Issue Type: New Feature
Reporter: Demai Ni
Assignee: Demai Ni
Fix For: 0.992
Attachments:
HBASE-11085-trunk-v1-contains-HBASE-10900-trunk-v4.patch,
HBASE-11085-trunk-v1.patch,
HBASE-11085-trunk-v2-contain-HBASE-10900-trunk-v4.patch,
HBASE-11085-trunk-v2.patch, HLogPlayer.java
h2. Feature Description
the jira is part of
[HBASE-7912|https://issues.apache.org/jira/browse/HBASE-7912], and depend on
full backup [HBASE-10900| https://issues.apache.org/jira/browse/HBASE-10900].
for the detail layout and frame work, please reference to [HBASE-10900|
https://issues.apache.org/jira/browse/HBASE-10900].
When client issues an incremental backup request, BackupManager will check
the request and then kicks of a global procedure via HBaseAdmin for all the
active regionServer to roll log. Each Region server will record their log
number into zookeeper. Then we determine which log need to be included in
this incremental backup, and use DistCp to copy them to target location. At
the same time, a dependency of backup image will be recorded, and later on
saved in Backup Manifest file.
Restore is to replay the backuped WAL logs on target HBase instance. The
replay will occur after full backup.
As incremental backup image depends on prior full backup image and
incremental images if exists. Manifest file will be used to store the
dependency lineage during backup, and used during restore time for PIT
restore.
h2. Use case(i.e example)
{code:title=Incremental Backup Restore example|borderStyle=solid}
/***/
/* STEP1: FULL backup from sourcecluster to targetcluster
/* if no table name specified, all tables from source cluster will be
backuped
/***/
[sourcecluster]$ hbase backup create full
hdfs://hostname.targetcluster.org:9000/userid/backupdir t1_dn,t2_dn,t3_dn
...
14/05/09 13:35:46 INFO backup.BackupManager: Backup request
backup_1399667695966 has been executed.
/***/
/* STEP2: In HBase Shell, put a few rows
/***/
hbase(main):002:0 put 't1_dn','row100','cf1:q1','value100_0509_increm1'
hbase(main):003:0 put 't2_dn','row100','cf1:q1','value100_0509_increm1'
hbase(main):004:0 put 't3_dn','row100','cf1:q1','value100_0509_increm1'
/***/
/* STEP3: Take the 1st incremental backup
/***/
[sourcecluster]$ hbase backup create incremental
hdfs://hostname.targetcluster.org:9000/userid/backupdir
...
14/05/09 13:37:45 INFO backup.BackupManager: Backup request
backup_1399667851020 has been executed.
/***/
/* STEP4: In HBase Shell, put a few more rows.
/* update 'row100', and create new 'row101'
/***/
hbase(main):005:0 put 't3_dn','row100','cf1:q1','value101_0509_increm2'
hbase(main):006:0 put 't2_dn','row100','cf1:q1','value101_0509_increm2'
hbase(main):007:0 put 't1_dn','row100','cf1:q1','value101_0509_increm2'
hbase(main):009:0 put 't1_dn','row101','cf1:q1','value101_0509_increm2'
hbase(main):010:0 put 't2_dn','row101','cf1:q1','value101_0509_increm2'
hbase(main):011:0 put 't3_dn','row101','cf1:q1','value101_0509_increm2'
/***/
/* STEP5: Take the 2nd incremental backup
/***/
[sourcecluster]$ hbase backup create incremental
hdfs://hostname.targetcluster.org:9000/userid/backupdir
...
14/05/09 13:39:33 INFO backup.BackupManager: Backup request
backup_1399667959165 has been executed.
[jira] [Updated] (HBASE-10900) FULL table backup and restore
[
https://issues.apache.org/jira/browse/HBASE-10900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-10900:
--
Fix Version/s: (was: 0.99.1)
0.99.2
FULL table backup and restore
-
Key: HBASE-10900
URL: https://issues.apache.org/jira/browse/HBASE-10900
Project: HBase
Issue Type: Task
Reporter: Demai Ni
Assignee: Demai Ni
Fix For: 0.99.2
Attachments: HBASE-10900-fullbackup-trunk-v1.patch,
HBASE-10900-trunk-v2.patch, HBASE-10900-trunk-v3.patch,
HBASE-10900-trunk-v4.patch
h2. Feature Description
This is a subtask of
[HBase-7912|https://issues.apache.org/jira/browse/HBASE-7912] to support FULL
backup/restore, and will complete the following function:
{code:title=Backup Restore example|borderStyle=solid}
/* backup from sourcecluster to targetcluster
*/
/* if no table name specified, all tables from source cluster will be
backuped */
[sourcecluster]$ hbase backup create full
hdfs://hostname.targetcluster.org:9000/userid/backupdir t1_dn,t2_dn,t3_dn
/* restore on targetcluser, this is a local restore
*/
/* backup_1396650096738 - backup image name
*/
/* t1_dn,etc are the original table names. All tables will be restored if not
specified */
/* t1_dn_restore, etc. are the restored table. if not specified, orginal
table name will be used*/
[targetcluster]$ hbase restore /userid/backupdir backup_1396650096738
t1_dn,t2_dn,t3_dn t1_dn_restore,t2_dn_restore,t3_dn_restore
/* restore from targetcluster back to source cluster, this is a remote restore
[sourcecluster]$ hbase restore
hdfs://hostname.targetcluster.org:9000/userid/backupdir backup_1396650096738
t1_dn,t2_dn,t3_dn t1_dn_restore,t2_dn_restore,t3_dn_restore
{code}
h2. Detail layout and frame work for the next jiras
The patch is a wrapper of the existing snapshot and exportSnapshot, and will
use as the base framework for the over-all solution of
[HBase-7912|https://issues.apache.org/jira/browse/HBASE-7912] as described
below:
* *bin/hbase* : end-user command line interface to invoke
BackupClient and RestoreClient
* *BackupClient.java* : 'main' entry for backup operations. This patch will
only support 'full' backup. In future jiras, will support:
** *create* incremental backup
** *cancel* an ongoing backup
** *delete* an exisitng backup image
** *describe* the detailed informaiton of backup image
** show *history* of all successful backups
** show the *status* of the latest backup request
** *convert* incremental backup WAL files into HFiles. either on-the-fly
during create or after create
** *merge* backup image
** *stop* backup a table of existing backup image
** *show* tables of a backup image
* *BackupCommands.java* : a place to keep all the command usages and options
* *BackupManager.java* : handle backup requests on server-side, create
BACKUP ZOOKEEPER nodes to keep track backup. The timestamps kept in zookeeper
will be used for future incremental backup (not included in this jira).
Create BackupContext and DispatchRequest.
* *BackupHandler.java* : in this patch, it is a wrapper of snapshot and
exportsnapshot. In future jiras,
** *timestamps* info will be recorded in ZK
** carry on *incremental* backup.
** update backup *progress*
** set flags of *status*
** build up *backupManifest* file(in this jira only limited info for
fullback. later on, timestamps and dependency of multipl backup images are
also recorded here)
** clean up after *failed* backup
** clean up after *cancelled* backup
** allow on-the-fly *convert* during incremental backup
* *BackupContext.java* : encapsulate backup information like backup ID, table
names, directory info, phase, TimeStamps of backup progress, size of data,
ancestor info, etc.
* *BackupCopier.java* : the copying operation. Later on, to support
progress report and mapper estimation; and extends DisCp for progress
updating to ZK during backup.
* *BackupExcpetion.java*: to handle exception from backup/restore
* *BackupManifest.java* : encapsulate all the backup image information. The
manifest info will be bundled as manifest file together with data. So that
each backup image will contain all the info needed for restore.
* *BackupStatus.java* : encapsulate backup status at table level during
backup progress
* *BackupUtil.java* : utility methods during backup process
* *RestoreClient.java* : 'main' entry for restore operations. This patch
will only support 'full' backup.
* *RestoreUtil.java*: utility methods during restore process
* *ExportSnapshot.java* : remove 'final' so that another class
[jira] [Updated] (HBASE-10800) Use CellComparator instead of KVComparator
[ https://issues.apache.org/jira/browse/HBASE-10800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10800: -- Fix Version/s: (was: 0.99.1) 0.99.2 Use CellComparator instead of KVComparator -- Key: HBASE-10800 URL: https://issues.apache.org/jira/browse/HBASE-10800 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.2 Attachments: HBASE-10800_1.patch, HBASE-10800_2.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-10671) Add missing InterfaceAudience annotations for classes in hbase-common and hbase-client modules
[ https://issues.apache.org/jira/browse/HBASE-10671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10671: -- Fix Version/s: (was: 0.99.1) 0.99.2 Add missing InterfaceAudience annotations for classes in hbase-common and hbase-client modules -- Key: HBASE-10671 URL: https://issues.apache.org/jira/browse/HBASE-10671 Project: HBase Issue Type: Sub-task Components: Client Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0, 0.99.2 Attachments: hbase-10671_v1.patch, hbase-10671_v2.patch In this jira, we'll add missing InterfaceAudience annotations to classes in the client visible modules (hbase-client and hbase-common). Parent jira is for deciding on whether some of the classes should be private or public. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11148) Provide a distributed procedure to globally roll logs
[
https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168382#comment-14168382
]
Hadoop QA commented on HBASE-11148:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12652564/HBASE-11148-trunk-v2.patch
against trunk revision .
ATTACHMENT ID: 12652564
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11315//console
This message is automatically generated.
Provide a distributed procedure to globally roll logs
-
Key: HBASE-11148
URL: https://issues.apache.org/jira/browse/HBASE-11148
Project: HBase
Issue Type: New Feature
Reporter: Jerry He
Assignee: Jerry He
Fix For: 2.0.0, 0.99.2
Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch
Propose a distributed procedure here to globally roll logs.
Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a
single RS.
Some use cases may require that all the RS roll the logs at the same time and
in a coordinated way. Also there may be requirement that some tasks be done
together with the roll log on each region server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-9431) Set 'hbase.bulkload.retries.number' to 10 as HBASE-8450 claims
[ https://issues.apache.org/jira/browse/HBASE-9431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-9431: - Attachment: hbase-12072_v1.patch I think this is the intended patch. Set 'hbase.bulkload.retries.number' to 10 as HBASE-8450 claims --- Key: HBASE-9431 URL: https://issues.apache.org/jira/browse/HBASE-9431 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.99.1 Attachments: 9431.txt, hbase-12072_v1.patch HBASE-8450 claimes 'hbase.bulkload.retries.number' is set to 10 when its still 0 ([~jeffreyz] noticed). Fix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12236) Change namespace of HTraceConfiguration dependency in 0.98
Ted Yu created HBASE-12236:
--
Summary: Change namespace of HTraceConfiguration dependency in 0.98
Key: HBASE-12236
URL: https://issues.apache.org/jira/browse/HBASE-12236
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
As discussed in thread 'NoSuchMethodError using zipkin with hbase 0.98.5',
HBaseSpanReceiver.config() method from htrace-hbase module expects parameter of
type org.htrace.HTraceConfiguration.
However, org.apache.hadoop.hbase.trace.HBaseHTraceConfiguration in 0.98 extends
org.cloudera.htrace.HTraceConfiguration , leading to the following compilation
error when building htrace-hbase against 0.98:
{code}
[ERROR]
/home/hadoop/git/htrace/htrace-hbase/src/main/java/org/htrace/impl/HBaseSpanReceiver.java:[341,12]
error: method configure in class HBaseSpanReceiver cannot be applied to
given types;
{code}
Thanks to Abhishek Kumar who reported the above issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-9003) TableMapReduceUtil should not rely on org.apache.hadoop.util.JarFinder#getJar
[
https://issues.apache.org/jira/browse/HBASE-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-9003:
-
Fix Version/s: (was: 0.99.1)
0.99.2
TableMapReduceUtil should not rely on org.apache.hadoop.util.JarFinder#getJar
-
Key: HBASE-9003
URL: https://issues.apache.org/jira/browse/HBASE-9003
Project: HBase
Issue Type: Bug
Components: mapreduce
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Fix For: 2.0.0, 0.99.2
Attachments: HBASE-9003.v0.patch, HBASE-9003.v1.patch,
HBASE-9003.v2.patch
This is the problem: {{TableMapReduceUtil#addDependencyJars}} relies on
{{org.apache.hadoop.util.JarFinder}} if available to call {{getJar()}}.
However {{getJar()}} uses File.createTempFile() to create a temporary file
under {{hadoop.tmp.dir}}{{/target/test-dir}}. Due HADOOP-9737 the created jar
and its content is not purged after the JVM is destroyed. Since most
configurations point {{hadoop.tmp.dir}} under {{/tmp}} the generated jar
files get purged by {{tmpwatch}} or a similar tool, but boxes that have
{{hadoop.tmp.dir}} pointing to a different location not monitored by
{{tmpwatch}} will pile up a collection of jars causing all kind of issues.
Since {{JarFinder#getJar}} is not a public API from Hadoop (see [~tucu00]
comment on HADOOP-9737) we shouldn't use that as part of
{{TableMapReduceUtil}} in order to avoid this kind of issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-8329) Limit compaction speed
[ https://issues.apache.org/jira/browse/HBASE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-8329: - Fix Version/s: (was: 0.99.1) 0.99.2 Limit compaction speed -- Key: HBASE-8329 URL: https://issues.apache.org/jira/browse/HBASE-8329 Project: HBase Issue Type: Improvement Components: Compaction Reporter: binlijin Assignee: Sergey Shelukhin Fix For: 2.0.0, 0.99.2 Attachments: HBASE-8329-2-trunk.patch, HBASE-8329-3-trunk.patch, HBASE-8329-4-trunk.patch, HBASE-8329-5-trunk.patch, HBASE-8329-6-trunk.patch, HBASE-8329-7-trunk.patch, HBASE-8329-8-trunk.patch, HBASE-8329-9-trunk.patch, HBASE-8329-trunk.patch There is no speed or resource limit for compaction，I think we should add this feature especially when request burst. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-8073) HFileOutputFormat support for offline operation
[ https://issues.apache.org/jira/browse/HBASE-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-8073: - Fix Version/s: (was: 0.99.1) 0.99.2 HFileOutputFormat support for offline operation --- Key: HBASE-8073 URL: https://issues.apache.org/jira/browse/HBASE-8073 Project: HBase Issue Type: Sub-task Components: mapreduce Reporter: Nick Dimiduk Assignee: Jerry He Fix For: 0.99.2 Attachments: HBASE-8073-trunk-v0.patch, HBASE-8073-trunk-v1.patch When using HFileOutputFormat to generate HFiles, it inspects the region topology of the target table. The split points from that table are used to guide the TotalOrderPartitioner. If the target table does not exist, it is first created. This imposes an unnecessary dependence on an online HBase and existing table. If the table exists, it can be used. However, the job can be smarter. For example, if there's far more data going into the HFiles than the table currently contains, the table regions aren't very useful for data split points. Instead, the input data can be sampled to produce split points more meaningful to the dataset. LoadIncrementalHFiles is already capable of handling divergence between HFile boundaries and table regions, so this should not pose any additional burdon at load time. The proper method of sampling the data likely requires a custom input format and an additional map-reduce job perform the sampling. See a relevant implementation: https://github.com/alexholmes/hadoop-book/blob/master/src/main/java/com/manning/hip/ch4/sampler/ReservoirSamplerInputFormat.java -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-8028) Append, Increment: Adding rollback support
[ https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-8028: - Fix Version/s: (was: 0.99.1) 0.99.2 Append, Increment: Adding rollback support -- Key: HBASE-8028 URL: https://issues.apache.org/jira/browse/HBASE-8028 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.5 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.99.2 Attachments: HBase-8028-v1.patch, HBase-8028-v2.patch, HBase-8028-with-Increments-v1.patch, HBase-8028-with-Increments-v2.patch In case there is an exception while doing the log-sync, the memstore is not rollbacked, while the mvcc is _always_ forwarded to the writeentry created at the beginning of the operation. This may lead to scanners seeing results which are not synched to the fs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-7115) [shell] Provide a way to register custom filters with the Filter Language Parser
[
https://issues.apache.org/jira/browse/HBASE-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-7115:
-
Fix Version/s: (was: 0.99.1)
0.99.2
[shell] Provide a way to register custom filters with the Filter Language
Parser
Key: HBASE-7115
URL: https://issues.apache.org/jira/browse/HBASE-7115
Project: HBase
Issue Type: Improvement
Components: Filters, shell
Affects Versions: 0.95.2
Reporter: Aditya Kishore
Assignee: Aditya Kishore
Fix For: 0.99.2
Attachments: HBASE-7115_trunk.patch, HBASE-7115_trunk.patch,
HBASE-7115_trunk_v2.patch
HBASE-5428 added this capability to thrift interface but the configuration
parameter name is thrift specific.
This patch introduces a more generic parameter hbase.user.filters using
which the user defined custom filters can be specified in the configuration
and loaded in any client that needs to use the filter language parser.
The patch then uses this new parameter to register any user specified filters
while invoking the HBase shell.
Example usage: Let's say I have written a couple of custom filters with class
names *{{org.apache.hadoop.hbase.filter.custom.SuperDuperFilter}}* and
*{{org.apache.hadoop.hbase.filter.custom.SilverBulletFilter}}* and I want to
use them from HBase shell using the filter language.
To do that, I would add the following configuration to {{hbase-site.xml}}
{panel}{{property}}
{{ namehbase.user.filters/name}}
{{
value}}*{{SuperDuperFilter}}*{{:org.apache.hadoop.hbase.filter.custom.SuperDuperFilter,}}*{{SilverBulletFilter}}*{{:org.apache.hadoop.hbase.filter.custom.SilverBulletFilter/value}}
{{/property}}{panel}
Once this is configured, I can launch HBase shell and use these filters in my
{{get}} or {{scan}} just the way I would use a built-in filter.
{code}
hbase(main):001:0 scan 't', {FILTER = SuperDuperFilter(true) AND
SilverBulletFilter(42)}
ROW COLUMN+CELL
status column=cf:a,
timestamp=30438552, value=world_peace
1 row(s) in 0. seconds
{code}
To use this feature in any client, the client needs to make the following
function call as part of its initialization.
{code}
ParseFilter.registerUserFilters(configuration);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-7320) Remove KeyValue.getBuffer()
[ https://issues.apache.org/jira/browse/HBASE-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-7320: - Fix Version/s: (was: 0.99.1) 0.99.2 Remove KeyValue.getBuffer() --- Key: HBASE-7320 URL: https://issues.apache.org/jira/browse/HBASE-7320 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Assignee: stack Fix For: 0.99.2 Attachments: 7320-simple.txt In many places this is simple task of just replacing the method name. There, however, quite a few places where we assume that: # the entire KV is backed by a single byte array # the KVs key portion is backed by a single byte array Some of those can easily be fixed, others will need their own jiras. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-5617) Provide coprocessor hooks in put flow while rollbackMemstore.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-5617: - Fix Version/s: (was: 0.99.1) 0.99.2 Provide coprocessor hooks in put flow while rollbackMemstore. - Key: HBASE-5617 URL: https://issues.apache.org/jira/browse/HBASE-5617 Project: HBase Issue Type: Improvement Components: Coprocessors Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 2.0.0, 0.99.2 Attachments: HBASE-5617_1.patch, HBASE-5617_2.patch With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false. In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked. Now the problem is that if the put that happens in the main flow fails there is no way to rollback the puts that happened in the prePut. We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-5583: - Fix Version/s: (was: 0.99.1) 0.99.2 Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.2 Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_1_review.patch, HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch, HBASE-5583_new_5_WIP_using_tableznode.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12012) Improve cancellation for the scan RPCs
[ https://issues.apache.org/jira/browse/HBASE-12012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-12012: -- Fix Version/s: (was: 0.99.1) 0.99.2 Improve cancellation for the scan RPCs -- Key: HBASE-12012 URL: https://issues.apache.org/jira/browse/HBASE-12012 Project: HBase Issue Type: Sub-task Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 2.0.0, 0.99.2 Attachments: 12012-1.txt Similar to HBASE-11564 but for scans. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12176) WALCellCodec Encoders support for non-KeyValue Cells
[ https://issues.apache.org/jira/browse/HBASE-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-12176: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed this to branch-1 and master. Thanks Anoop. WALCellCodec Encoders support for non-KeyValue Cells Key: HBASE-12176 URL: https://issues.apache.org/jira/browse/HBASE-12176 Project: HBase Issue Type: Improvement Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 2.0.0, 0.99.1 Attachments: HBASE-12176.patch Now in all Encoder returned by WALCellCodec and SecureWALCellCodec it is assumed that it will get only KeyValues. If not it just throws IOE. Now we support Cells through out in write path as well and these Encoders should support non KV cells as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-10974) Improve DBEs read performance by avoiding byte array deep copies for key[] and value[]
[ https://issues.apache.org/jira/browse/HBASE-10974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10974: -- Fix Version/s: (was: 0.99.1) 0.99.2 Improve DBEs read performance by avoiding byte array deep copies for key[] and value[] -- Key: HBASE-10974 URL: https://issues.apache.org/jira/browse/HBASE-10974 Project: HBase Issue Type: Improvement Components: Scanners Affects Versions: 0.99.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.2 Attachments: HBASE-10974_1.patch As part of HBASE-10801, we tried to reduce the copy of the value [] in forming the KV from the DBEs. The keys required copying and this was restricting us in using Cells and always wanted to copy to be done. The idea here is to replace the key byte[] as ByteBuffer and create a consecutive stream of the keys (currently the same byte[] is used and hence the copy). Use offset and length to track this key bytebuffer. The copy of the encoded format to normal Key format is definitely needed and can't be avoided but we could always avoid the deep copy of the bytes to form a KV and thus use cells effectively. Working on a patch, will post it soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-10154) Add a unit test for Canary tool
[ https://issues.apache.org/jira/browse/HBASE-10154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10154: -- Fix Version/s: (was: 0.99.1) 0.99.2 Add a unit test for Canary tool --- Key: HBASE-10154 URL: https://issues.apache.org/jira/browse/HBASE-10154 Project: HBase Issue Type: Improvement Components: monitoring, test Reporter: takeshi.miao Assignee: takeshi.miao Priority: Minor Fix For: 0.99.2 Attachments: HBASE-10154-trunk-v01.patch, HBASE-10154-trunk-v02.patch, HBASE-10154-trunk-v03.patch Due to HBASE-10108, I am working to come out a unit test for o.h.hbase.tool.Canary to eliminate this kind of issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12236) Change namespace of HTraceConfiguration dependency in 0.98
[
https://issues.apache.org/jira/browse/HBASE-12236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12236:
---
Attachment: 12236-v1.txt
TestHTraceHooks passes.
Change namespace of HTraceConfiguration dependency in 0.98
--
Key: HBASE-12236
URL: https://issues.apache.org/jira/browse/HBASE-12236
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 12236-v1.txt
As discussed in thread 'NoSuchMethodError using zipkin with hbase 0.98.5',
HBaseSpanReceiver.config() method from htrace-hbase module expects parameter
of type org.htrace.HTraceConfiguration.
However, org.apache.hadoop.hbase.trace.HBaseHTraceConfiguration in 0.98
extends org.cloudera.htrace.HTraceConfiguration , leading to the following
compilation error when building htrace-hbase against 0.98:
{code}
[ERROR]
/home/hadoop/git/htrace/htrace-hbase/src/main/java/org/htrace/impl/HBaseSpanReceiver.java:[341,12]
error: method configure in class HBaseSpanReceiver cannot be applied to
given types;
{code}
Thanks to Abhishek Kumar who reported the above issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-10200) Better error message when HttpServer fails to start due to java.net.BindException
[
https://issues.apache.org/jira/browse/HBASE-10200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-10200:
--
Fix Version/s: (was: 0.99.1)
0.99.2
Better error message when HttpServer fails to start due to
java.net.BindException
-
Key: HBASE-10200
URL: https://issues.apache.org/jira/browse/HBASE-10200
Project: HBase
Issue Type: Task
Affects Versions: 2.0.0
Reporter: Ted Yu
Assignee: Kiran Kumar M R
Priority: Minor
Labels: beginner
Fix For: 2.0.0, 0.99.2
Attachments: 10200.out, HBASE-10020-V2.patch, HBASE-10020-V3.patch,
HBASE-10020.patch, HBASE-10020.patch
Starting HBase using Hoya, I saw the following in log:
{code}
2013-12-17 21:49:06,758 INFO [master:hor12n19:42587] http.HttpServer:
HttpServer.start() threw a non Bind IOException
java.net.BindException: Port in use: hor12n14.gq1.ygridcore.net:12432
at org.apache.hadoop.http.HttpServer.openListener(HttpServer.java:742)
at org.apache.hadoop.http.HttpServer.start(HttpServer.java:686)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:586)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.net.BindException: Cannot assign requested address
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:344)
at sun.nio.ch.Net.bind(Net.java:336)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:199)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at
org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
at org.apache.hadoop.http.HttpServer.openListener(HttpServer.java:738)
{code}
This was due to hbase.master.info.bindAddress giving static address but Hoya
allocates master dynamically.
Better error message should be provided: when bindAddress points another host
than local host, message should remind user to remove / adjust
hbase.master.info.bindAddress config param from hbase-site.xml
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11699: -- Fix Version/s: (was: 0.99.1) 0.99.2 Region servers exclusion list to HMaster. - Key: HBASE-11699 URL: https://issues.apache.org/jira/browse/HBASE-11699 Project: HBase Issue Type: New Feature Components: Admin, Client, regionserver, Zookeeper Affects Versions: 0.98.3 Reporter: Gomathivinayagam Muthuvinayagam Priority: Minor Labels: patch Fix For: 2.0.0, 0.98.8, 0.99.2 Attachments: HBASE_11699.patch, HBASE_11699_v1.patch, HBASE_11699_v2.patch, HBASE_11699_v3.patch, HBASE_11699_v4.patch, HBASE_11699_v5.patch, HBASE_11699_v6.patch, HBASE_11699_v7.patch Original Estimate: 96h Remaining Estimate: 96h Currently HBase does not support adding set of region servers to be in the exclusion list. So that administrators can prevent accidental startups of some region servers to join the cluster. There was initially some work done, and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It was not done after that. I am planning to contribute it as a patch, and I would like to do some improvements as well. Instead of storing the exclusion entries on a file, I am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6618) Implement FuzzyRowFilter with ranges support
[ https://issues.apache.org/jira/browse/HBASE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-6618: - Fix Version/s: (was: 0.99.1) 0.99.2 Implement FuzzyRowFilter with ranges support Key: HBASE-6618 URL: https://issues.apache.org/jira/browse/HBASE-6618 Project: HBase Issue Type: New Feature Components: Filters Reporter: Alex Baranau Assignee: Alex Baranau Priority: Minor Fix For: 0.99.2 Attachments: HBASE-6618-algo-desc-bits.png, HBASE-6618-algo.patch, HBASE-6618.patch, HBASE-6618_2.path, HBASE-6618_3.path, HBASE-6618_4.patch, HBASE-6618_5.patch Apart from current ability to specify fuzzy row filter e.g. for userId_actionId format as _0004 (where 0004 - actionId) it would be great to also have ability to specify the fuzzy range , e.g. _0004, ..., _0099. See initial discussion here: http://search-hadoop.com/m/WVLJdX0Z65 Note: currently it is possible to provide multiple fuzzy row rules to existing FuzzyRowFilter, but in case when the range is big (contains thousands of values) it is not efficient. Filter should perform efficient fast-forwarding during the scan (this is what distinguishes it from regex row filter). While such functionality may seem like a proper fit for custom filter (i.e. not including into standard filter set) it looks like the filter may be very re-useable. We may judge based on the implementation that will hopefully be added. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-10602) Cleanup HTable public interface
[ https://issues.apache.org/jira/browse/HBASE-10602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10602: -- Fix Version/s: (was: 0.99.1) 0.99.2 Cleanup HTable public interface --- Key: HBASE-10602 URL: https://issues.apache.org/jira/browse/HBASE-10602 Project: HBase Issue Type: Umbrella Components: Client, Usability Reporter: Nick Dimiduk Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.2 Attachments: hbase-10602_v1.patch HBASE-6580 replaced the preferred means of HTableInterface acquisition to the HConnection#getTable factory methods. HBASE-9117 removes the HConnection cache, placing the burden of responsible connection cleanup on whomever acquires it. The remaining HTable constructors use a Connection instance and manage their own HConnection on the callers behalf. This is convenient but also a surprising source of poor performance for anyone accustomed to the previous connection caching behavior. I propose deprecating those remaining constructors for 0.98/0.96 and removing them for 1.0. While I'm at it, I suggest we pursue some API hygiene in general and convert HTable into an interface. I'm sure there are method overloads for accepting String/byte[]/TableName where just TableName is sufficient. Can that be done for 1.0 as well? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12233) Bring back root table
[ https://issues.apache.org/jira/browse/HBASE-12233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168402#comment-14168402 ] stack commented on HBASE-12233: --- [~apurtell] I like your suggestion of a feature branch. Bring back root table - Key: HBASE-12233 URL: https://issues.apache.org/jira/browse/HBASE-12233 Project: HBase Issue Type: Sub-task Reporter: Virag Kothari Assignee: Virag Kothari Attachments: HBASE-12233.patch First step towards splitting meta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12072) We are doing 35 x 35 retries for master operations
[
https://issues.apache.org/jira/browse/HBASE-12072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168408#comment-14168408
]
Hadoop QA commented on HBASE-12072:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12674380/hbase-12072_v1.patch
against trunk revision .
ATTACHMENT ID: 12674380
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 15 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11314//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11314//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11314//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11314//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11314//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11314//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11314//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11314//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11314//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11314//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11314//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11314//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11314//artifact/trunk/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11314//console
This message is automatically generated.
We are doing 35 x 35 retries for master operations
--
Key: HBASE-12072
URL: https://issues.apache.org/jira/browse/HBASE-12072
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.6
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Attachments: 12072-v1.txt, 12072-v2.txt, hbase-12072_v1.patch
For master requests, there are two retry mechanisms in effect. The first one
is from HBaseAdmin.executeCallable()
{code}
private V V executeCallable(MasterCallableV callable) throws
IOException {
RpcRetryingCallerV caller = rpcCallerFactory.newCaller();
try {
return caller.callWithRetries(callable);
} finally {
callable.close();
}
}
{code}
And inside, the other one is from StubMaker.makeStub():
{code}
/**
* Create a stub against the master. Retry if necessary.
* @return A stub to do codeintf/code against the master
* @throws MasterNotRunningException
*/
@edu.umd.cs.findbugs.annotations.SuppressWarnings
(value=SWL_SLEEP_WITH_LOCK_HELD)
Object makeStub() throws MasterNotRunningException {
{code}
The tests will just hang for 10 min * 35 ~= 6hours.
{code}
2014-09-23 16:19:05,151 INFO [main]
client.ConnectionManager$HConnectionImplementation: getMaster attempt 1 of 35
failed; retrying after sleep of 100, exception=java.io.IOException: Can't get
master address from ZooKeeper; znode data == null
2014-09-23 16:19:05,253
[jira] [Commented] (HBASE-9431) Set 'hbase.bulkload.retries.number' to 10 as HBASE-8450 claims
[
https://issues.apache.org/jira/browse/HBASE-9431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168414#comment-14168414
]
Hadoop QA commented on HBASE-9431:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12674381/hbase-12072_v1.patch
against trunk revision .
ATTACHMENT ID: 12674381
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 15 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11316//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11316//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11316//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11316//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11316//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11316//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11316//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11316//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11316//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11316//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11316//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11316//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11316//artifact/trunk/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11316//console
This message is automatically generated.
Set 'hbase.bulkload.retries.number' to 10 as HBASE-8450 claims
---
Key: HBASE-9431
URL: https://issues.apache.org/jira/browse/HBASE-9431
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: stack
Fix For: 0.99.1
Attachments: 9431.txt, hbase-12072_v1.patch
HBASE-8450 claimes 'hbase.bulkload.retries.number' is set to 10 when its
still 0 ([~jeffreyz] noticed). Fix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12176) WALCellCodec Encoders support for non-KeyValue Cells
[ https://issues.apache.org/jira/browse/HBASE-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168423#comment-14168423 ] Hudson commented on HBASE-12176: SUCCESS: Integrated in HBase-TRUNK #5650 (See [https://builds.apache.org/job/HBase-TRUNK/5650/]) HBASE-12176 WALCellCodec Encoders support for non-KeyValue Cells (Anoop Sam John) (enis: rev e3f99578304cb30f1b59c34a4193b9b700f3566a) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SecureWALCellCodec.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AuthResult.java WALCellCodec Encoders support for non-KeyValue Cells Key: HBASE-12176 URL: https://issues.apache.org/jira/browse/HBASE-12176 Project: HBase Issue Type: Improvement Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 2.0.0, 0.99.1 Attachments: HBASE-12176.patch Now in all Encoder returned by WALCellCodec and SecureWALCellCodec it is assumed that it will get only KeyValues. If not it just throws IOE. Now we support Cells through out in write path as well and these Encoders should support non KV cells as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12176) WALCellCodec Encoders support for non-KeyValue Cells
[ https://issues.apache.org/jira/browse/HBASE-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168426#comment-14168426 ] Hudson commented on HBASE-12176: SUCCESS: Integrated in HBase-1.0 #299 (See [https://builds.apache.org/job/HBase-1.0/299/]) HBASE-12176 WALCellCodec Encoders support for non-KeyValue Cells (Anoop Sam John) (enis: rev 8938dc96306c6ea96c050dd0b8837dbb65985031) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SecureWALCellCodec.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AuthResult.java WALCellCodec Encoders support for non-KeyValue Cells Key: HBASE-12176 URL: https://issues.apache.org/jira/browse/HBASE-12176 Project: HBase Issue Type: Improvement Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 2.0.0, 0.99.1 Attachments: HBASE-12176.patch Now in all Encoder returned by WALCellCodec and SecureWALCellCodec it is assumed that it will get only KeyValues. If not it just throws IOE. Now we support Cells through out in write path as well and these Encoders should support non KV cells as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11764) Support per cell TTLs
[ https://issues.apache.org/jira/browse/HBASE-11764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11764: -- Fix Version/s: (was: 0.99.1) 0.99.2 Support per cell TTLs - Key: HBASE-11764 URL: https://issues.apache.org/jira/browse/HBASE-11764 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.8, 0.99.2 Attachments: HBASE-11764-0.98.patch, HBASE-11764-0.98.patch, HBASE-11764-0.98.patch, HBASE-11764-0.98.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11590) use a specific ThreadPoolExecutor
[ https://issues.apache.org/jira/browse/HBASE-11590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11590: -- Fix Version/s: (was: 0.99.1) 0.99.2 use a specific ThreadPoolExecutor - Key: HBASE-11590 URL: https://issues.apache.org/jira/browse/HBASE-11590 Project: HBase Issue Type: Bug Components: Client, Performance Affects Versions: 1.0.0, 2.0.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 2.0.0, 0.99.2 Attachments: tp.patch The JDK TPE creates all the threads in the pool. As a consequence, we create (by default) 256 threads even if we just need a few. The attached TPE create threads only if we have something in the queue. On a PE test with replica on, it improved the 99 latency percentile by 5%. Warning: there are likely some race conditions, but I'm posting it here because there is may be an implementation available somewhere we can use, or a good reason not to do that. So feedback welcome as usual. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-8642) [Snapshot] List and delete snapshot by table
[ https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-8642: - Fix Version/s: (was: 0.99.1) 0.99.2 [Snapshot] List and delete snapshot by table Key: HBASE-8642 URL: https://issues.apache.org/jira/browse/HBASE-8642 Project: HBase Issue Type: Improvement Components: snapshots Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Reporter: Julian Zhou Assignee: Julian Zhou Priority: Minor Fix For: 0.99.2 Attachments: 8642-trunk-0.95-v0.patch, 8642-trunk-0.95-v1.patch, 8642-trunk-0.95-v2.patch Support list and delete snapshot by table name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table
[ https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168444#comment-14168444 ] Enis Soztutar commented on HBASE-8642: -- Lets do this. My only concern is that all new API should only be using TableName args, and not byte[] or String table name args. [Snapshot] List and delete snapshot by table Key: HBASE-8642 URL: https://issues.apache.org/jira/browse/HBASE-8642 Project: HBase Issue Type: Improvement Components: snapshots Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Reporter: Julian Zhou Assignee: Julian Zhou Priority: Minor Fix For: 0.99.2 Attachments: 8642-trunk-0.95-v0.patch, 8642-trunk-0.95-v1.patch, 8642-trunk-0.95-v2.patch Support list and delete snapshot by table name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8533) HBaseAdmin does not ride over cluster restart
[ https://issues.apache.org/jira/browse/HBASE-8533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168445#comment-14168445 ] Enis Soztutar commented on HBASE-8533: -- Would the retrying in HBASE-12072 help in this case? HBaseAdmin does not ride over cluster restart - Key: HBASE-8533 URL: https://issues.apache.org/jira/browse/HBASE-8533 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.0, 0.95.0 Reporter: Julian Zhou Assignee: Julian Zhou Priority: Minor Fix For: 0.99.2 Attachments: 8533-0.95-v1.patch, 8533-trunk-v1.patch, hbase-8533-trunk-v0.patch For Restful servlet (org.apache.hadoop.hbase.rest.Main (0.94), org.apache.hadoop.hbase.rest.RESTServer (trunk)) on Jetty, we need to first explicitly start the service (% ./bin/hbase-daemon.sh start rest -p 8000 ) for application running. Here is a scenario, sometimes, HBase cluster are stopped/started for maintanence, but rest is a seperated standalone process, which binds the HBaseAdmin at construction method. HBase stop/start cause this binding lost for existing rest servlet. Rest servlet still exist to trying on old bound HBaseAdmin until a long time duration later with an Unavailable caught via an IOException caught in such as RootResource. Could we pairwise the HBase service with HBase rest service with some start/stop options? since seems no reason to still keep the rest servlet process after HBase stopped? When HBase restarts, original rest service could not resume to bind to the new HBase service via its old HBaseAdmin reference? So may we stop the rest when hbase stopped, or even if hbase was killed by acident, restart hbase with rest option could detect the old rest process, kill it and start to bind a new one? From this point of view, application rely on rest api in previous scenario could immediately detect it when setting up http connection session instead of wasting a long time to fail back from IOException with Unavailable from rest servlet. Put current options from the discussion history here from Andrew, Stack and Jean-Daniel, 1) create an HBaseAdmin on demand in rest servlet instead of keeping singleton instance; (another possible enhancement for HBase client: automatic reconnection of an open HBaseAdmin handle after a cluster bounce?) 2) pairwise the rest webapp with hbase webui so the rest is always on with HBase serive; 3) add an option for rest service (such as HBASE_MANAGES_REST) in hbase-env.sh, set HBASE_MANAGES_REST to true, the scripts will start/stop the REST server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-8533) HBaseAdmin does not ride over cluster restart
[ https://issues.apache.org/jira/browse/HBASE-8533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-8533: - Fix Version/s: (was: 0.99.1) 0.99.2 HBaseAdmin does not ride over cluster restart - Key: HBASE-8533 URL: https://issues.apache.org/jira/browse/HBASE-8533 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.0, 0.95.0 Reporter: Julian Zhou Assignee: Julian Zhou Priority: Minor Fix For: 0.99.2 Attachments: 8533-0.95-v1.patch, 8533-trunk-v1.patch, hbase-8533-trunk-v0.patch For Restful servlet (org.apache.hadoop.hbase.rest.Main (0.94), org.apache.hadoop.hbase.rest.RESTServer (trunk)) on Jetty, we need to first explicitly start the service (% ./bin/hbase-daemon.sh start rest -p 8000 ) for application running. Here is a scenario, sometimes, HBase cluster are stopped/started for maintanence, but rest is a seperated standalone process, which binds the HBaseAdmin at construction method. HBase stop/start cause this binding lost for existing rest servlet. Rest servlet still exist to trying on old bound HBaseAdmin until a long time duration later with an Unavailable caught via an IOException caught in such as RootResource. Could we pairwise the HBase service with HBase rest service with some start/stop options? since seems no reason to still keep the rest servlet process after HBase stopped? When HBase restarts, original rest service could not resume to bind to the new HBase service via its old HBaseAdmin reference? So may we stop the rest when hbase stopped, or even if hbase was killed by acident, restart hbase with rest option could detect the old rest process, kill it and start to bind a new one? From this point of view, application rely on rest api in previous scenario could immediately detect it when setting up http connection session instead of wasting a long time to fail back from IOException with Unavailable from rest servlet. Put current options from the discussion history here from Andrew, Stack and Jean-Daniel, 1) create an HBaseAdmin on demand in rest servlet instead of keeping singleton instance; (another possible enhancement for HBase client: automatic reconnection of an open HBaseAdmin handle after a cluster bounce?) 2) pairwise the rest webapp with hbase webui so the rest is always on with HBase serive; 3) add an option for rest service (such as HBASE_MANAGES_REST) in hbase-env.sh, set HBASE_MANAGES_REST to true, the scripts will start/stop the REST server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-9531) a command line (hbase shell) interface to retreive the replication metrics and show replication lag
[
https://issues.apache.org/jira/browse/HBASE-9531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-9531:
-
Fix Version/s: (was: 0.99.1)
0.99.2
a command line (hbase shell) interface to retreive the replication metrics
and show replication lag
---
Key: HBASE-9531
URL: https://issues.apache.org/jira/browse/HBASE-9531
Project: HBase
Issue Type: New Feature
Components: Replication
Affects Versions: 0.99.0
Reporter: Demai Ni
Assignee: Demai Ni
Fix For: 2.0.0, 0.98.8, 0.99.2
Attachments: HBASE-9531-master-v1.patch, HBASE-9531-master-v1.patch,
HBASE-9531-master-v1.patch, HBASE-9531-master-v2.patch,
HBASE-9531-master-v3.patch, HBASE-9531-master-v4.patch,
HBASE-9531-trunk-v0.patch, HBASE-9531-trunk-v0.patch
This jira is to provide a command line (hbase shell) interface to retreive
the replication metrics info such as:ageOfLastShippedOp,
timeStampsOfLastShippedOp, sizeOfLogQueue ageOfLastAppliedOp, and
timeStampsOfLastAppliedOp. And also to provide a point of time info of the
lag of replication(source only)
Understand that hbase is using Hadoop
metrics(http://hbase.apache.org/metrics.html), which is a common way to
monitor metric info. This Jira is to serve as a light-weight client
interface, comparing to a completed(certainly better, but heavier)GUI
monitoring package. I made the code works on 0.94.9 now, and like to use this
jira to get opinions about whether the feature is valuable to other
users/workshop. If so, I will build a trunk patch.
All inputs are greatly appreciated. Thank you!
The overall design is to reuse the existing logic which supports hbase shell
command 'status', and invent a new module, called ReplicationLoad. In
HRegionServer.buildServerLoad() , use the local replication service objects
to get their loads which could be wrapped in a ReplicationLoad object and
then simply pass it to the ServerLoad. In ReplicationSourceMetrics and
ReplicationSinkMetrics, a few getters and setters will be created, and ask
Replication to build a ReplicationLoad. (many thanks to Jean-Daniel for
his kindly suggestions through dev email list)
the replication lag will be calculated for source only, and use this formula:
{code:title=Replication lag|borderStyle=solid}
if sizeOfLogQueue != 0 then max(ageOfLastShippedOp, (current time -
timeStampsOfLastShippedOp)) //err on the large side
else if (current time - timeStampsOfLastShippedOp) 2*
ageOfLastShippedOp then lag = ageOfLastShippedOp // last shipped happen
recently
else lag = 0 // last shipped may happens last night, so NO real lag
although ageOfLastShippedOp is non-zero
{code}
External will look something like:
{code:title=status 'replication'|borderStyle=solid}
hbase(main):001:0 status 'replication'
version 0.94.9
3 live servers
hdtest017.svl.ibm.com:
SOURCE:PeerID=1, ageOfLastShippedOp=14, sizeOfLogQueue=0,
timeStampsOfLastShippedOp=Wed Sep 04 14:49:48 PDT 2013
SINK :AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Wed Sep 04
14:48:48 PDT 2013
hdtest018.svl.ibm.com:
SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0,
timeStampsOfLastShippedOp=Wed Sep 04 14:48:48 PDT 2013
SINK :AgeOfLastAppliedOp=14, TimeStampsOfLastAppliedOp=Wed Sep 04
14:50:59 PDT 2013
hdtest015.svl.ibm.com:
SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0,
timeStampsOfLastShippedOp=Wed Sep 04 14:48:48 PDT 2013
SINK :AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Wed Sep 04
14:48:48 PDT 2013
hbase(main):002:0 status 'replication','source'
version 0.94.9
3 live servers
hdtest017.svl.ibm.com:
SOURCE:PeerID=1, ageOfLastShippedOp=14, sizeOfLogQueue=0,
timeStampsOfLastShippedOp=Wed Sep 04 14:49:48 PDT 2013
hdtest018.svl.ibm.com:
SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0,
timeStampsOfLastShippedOp=Wed Sep 04 14:48:48 PDT 2013
hdtest015.svl.ibm.com:
SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0,
timeStampsOfLastShippedOp=Wed Sep 04 14:48:48 PDT 2013
hbase(main):003:0 status 'replication','sink'
version 0.94.9
3 live servers
hdtest017.svl.ibm.com:
SINK :AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Wed Sep 04
14:48:48 PDT 2013
hdtest018.svl.ibm.com:
SINK :AgeOfLastAppliedOp=14, TimeStampsOfLastAppliedOp=Wed Sep 04
14:50:59 PDT 2013
hdtest015.svl.ibm.com:
SINK :AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Wed Sep 04
14:48:48 PDT 2013
hbase(main):003:0 status 'replication','lag'
version 0.94.9
3 live servers
hdtest017.svl.ibm.com: lag = 0
[jira] [Updated] (HBASE-6562) Fake KVs are sometimes passed to filters
[ https://issues.apache.org/jira/browse/HBASE-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-6562: - Fix Version/s: (was: 0.99.1) 0.99.2 Fake KVs are sometimes passed to filters Key: HBASE-6562 URL: https://issues.apache.org/jira/browse/HBASE-6562 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.99.2 Attachments: 6562-0.94-v1.txt, 6562-0.96-v1.txt, 6562-v2.txt, 6562-v3.txt, 6562-v4.txt, 6562-v5.txt, 6562.txt, minimalTest.java In internal tests at Salesforce we found that fake row keys sometimes are passed to filters (Filter.filterRowKey(...) specifically). The KVs are eventually filtered by the StoreScanner/ScanQueryMatcher, but the row key is passed to filterRowKey in RegionScannImpl *before* that happens. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9083) Downstreamers have to include a load of runtime dependencies
[
https://issues.apache.org/jira/browse/HBASE-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168448#comment-14168448
]
Enis Soztutar commented on HBASE-9083:
--
Is this still a problem?
Downstreamers have to include a load of runtime dependencies
Key: HBASE-9083
URL: https://issues.apache.org/jira/browse/HBASE-9083
Project: HBase
Issue Type: Sub-task
Components: build
Reporter: stack
Labels: beginner
Fix For: 0.99.2
Attachments: HBASE_9083.patch
Here is example from 0.95. Downstream project includes hbase-client ONLY.
To run the downstream project, here are the runtime dependencies currently.
This is hadoop1.
{code}
java -cp
target/client-1.0-SNAPSHOT.jar:/Users/stack/.m2/repository/org/apache/hbase/hbase-client/0.95.2-hadoop1-SNAPSHOT/hbase-client-0.95.2-hadoop1-SNAPSHOT.jar:/Users/stack/.m2/repository/org/apache/hbase/hbase-common/0.95.2-hadoop1-SNAPSHOT/hbase-common-0.95.2-hadoop1-SNAPSHOT.jar:/Users/stack/.m2/repository/org/apache/hadoop/hadoop-core/1.1.2/hadoop-core-1.1.2.jar:/Users/stack/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar:/Users/stack/.m2/repository/com/google/protobuf/protobuf-java/2.4.1/protobuf-java-2.4.1.jar:/Users/stack/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar:/Users/stack/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/Users/stack/.m2/repository/org/apache/hbase/hbase-protocol/0.95.2-hadoop1-SNAPSHOT/hbase-protocol-0.95.2-hadoop1-SNAPSHOT.jar:/Users/stack/.m2/repository/org/apache/zookeeper/zookeeper/3.4.5/zookeeper-3.4.5.jar:/Users/stack/.m2/repository/org/slf4j/slf4j-api/1.6.4/slf4j-api-1.6.4.jar:/Users/stack/.m2/repository/com/google/guava/guava/12.0.1/guava-12.0.1.jar:/Users/stack/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.8.8/jackson-mapper-asl-1.8.8.jar:/Users/stack/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.8.8/jackson-core-asl-1.8.8.jar:/Users/stack/.m2/repository/org/cloudera/htrace/htrace/1.50/htrace-1.50.jar:/Users/stack/.m2/repository/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar:/Users/stack/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar
org.hbase.downstream.Client
{code}
Thats:
{code}
hbase-client
base-common
hbase-protocol
hadoop-core
commons-logging
protobuf
commons-lang
commons-configuration
zookeeper
slf4j-api (AND commons-logging!)
guava
jackson-mapper-asl
jackson-core-asl
htrace
slf4j-log4j12
slf4j
{code}
Most of the above come in because of hadoop and zk (zk wants slf4j).
Can we shed any of these?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-9083) Downstreamers have to include a load of runtime dependencies
[
https://issues.apache.org/jira/browse/HBASE-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-9083:
-
Fix Version/s: (was: 0.99.1)
0.99.2
Downstreamers have to include a load of runtime dependencies
Key: HBASE-9083
URL: https://issues.apache.org/jira/browse/HBASE-9083
Project: HBase
Issue Type: Sub-task
Components: build
Reporter: stack
Labels: beginner
Fix For: 0.99.2
Attachments: HBASE_9083.patch
Here is example from 0.95. Downstream project includes hbase-client ONLY.
To run the downstream project, here are the runtime dependencies currently.
This is hadoop1.
{code}
java -cp
target/client-1.0-SNAPSHOT.jar:/Users/stack/.m2/repository/org/apache/hbase/hbase-client/0.95.2-hadoop1-SNAPSHOT/hbase-client-0.95.2-hadoop1-SNAPSHOT.jar:/Users/stack/.m2/repository/org/apache/hbase/hbase-common/0.95.2-hadoop1-SNAPSHOT/hbase-common-0.95.2-hadoop1-SNAPSHOT.jar:/Users/stack/.m2/repository/org/apache/hadoop/hadoop-core/1.1.2/hadoop-core-1.1.2.jar:/Users/stack/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar:/Users/stack/.m2/repository/com/google/protobuf/protobuf-java/2.4.1/protobuf-java-2.4.1.jar:/Users/stack/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar:/Users/stack/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/Users/stack/.m2/repository/org/apache/hbase/hbase-protocol/0.95.2-hadoop1-SNAPSHOT/hbase-protocol-0.95.2-hadoop1-SNAPSHOT.jar:/Users/stack/.m2/repository/org/apache/zookeeper/zookeeper/3.4.5/zookeeper-3.4.5.jar:/Users/stack/.m2/repository/org/slf4j/slf4j-api/1.6.4/slf4j-api-1.6.4.jar:/Users/stack/.m2/repository/com/google/guava/guava/12.0.1/guava-12.0.1.jar:/Users/stack/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.8.8/jackson-mapper-asl-1.8.8.jar:/Users/stack/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.8.8/jackson-core-asl-1.8.8.jar:/Users/stack/.m2/repository/org/cloudera/htrace/htrace/1.50/htrace-1.50.jar:/Users/stack/.m2/repository/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar:/Users/stack/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar
org.hbase.downstream.Client
{code}
Thats:
{code}
hbase-client
base-common
hbase-protocol
hadoop-core
commons-logging
protobuf
commons-lang
commons-configuration
zookeeper
slf4j-api (AND commons-logging!)
guava
jackson-mapper-asl
jackson-core-asl
htrace
slf4j-log4j12
slf4j
{code}
Most of the above come in because of hadoop and zk (zk wants slf4j).
Can we shed any of these?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-9864) Notifications bus for use by cluster members keeping up-to-date on changes
[ https://issues.apache.org/jira/browse/HBASE-9864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-9864: - Fix Version/s: (was: 0.99.1) 0.99.2 Notifications bus for use by cluster members keeping up-to-date on changes -- Key: HBASE-9864 URL: https://issues.apache.org/jira/browse/HBASE-9864 Project: HBase Issue Type: Brainstorming Reporter: stack Priority: Blocker Fix For: 0.99.2 In namespaces and acls, zk callbacks are used so all participating servers are notified when there is a change in acls/namespaces list. The new visibility tags feature coming in copies the same model of using zk with listeners for the features' particular notifications. Three systems each w/ their own implementation of the notifications all using zk w/ their own feature-specific watchers. Should probably unify. Do we have to go via zk? Seems like all want to be notified when an hbase table is updated. Could we tell servers directly rather than go via zk? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11915) Document and test 0.94 - 1.0.0 update
[ https://issues.apache.org/jira/browse/HBASE-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11915: -- Fix Version/s: (was: 0.99.1) 0.99.2 Document and test 0.94 - 1.0.0 update -- Key: HBASE-11915 URL: https://issues.apache.org/jira/browse/HBASE-11915 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Priority: Critical Fix For: 0.99.2 We explicitly did not remove some of the upgrade related stuff in branch-1 for the possibility of supporting 0.94 - 1.0, similar to 0.94 - 0.98 support. We should document, and test this support before 1.0 comes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-10499) In write heavy scenario one of the regions does not get flushed causing RegionTooBusyException
[
https://issues.apache.org/jira/browse/HBASE-10499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-10499:
--
Fix Version/s: (was: 0.99.1)
0.99.2
In write heavy scenario one of the regions does not get flushed causing
RegionTooBusyException
--
Key: HBASE-10499
URL: https://issues.apache.org/jira/browse/HBASE-10499
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Fix For: 0.99.2
Attachments: HBASE-10499.patch, compaction-queue.png,
hbase-root-master-ip-10-157-0-229.zip,
hbase-root-regionserver-ip-10-93-128-92.zip, master_4e39.log,
master_576f.log, rs_4e39.log, rs_576f.log, t1.dump, t2.dump,
workloada_0.98.dat
I got this while testing 0.98RC. But am not sure if it is specific to this
version. Doesn't seem so to me.
Also it is something similar to HBASE-5312 and HBASE-5568.
Using 10 threads i do writes to 4 RS using YCSB. The table created has 200
regions. In one of the run with 0.98 server and 0.98 client I faced this
problem like the hlogs became more and the system requested flushes for those
many regions.
One by one everything was flushed except one and that one thing remained
unflushed. The ripple effect of this on the client side
{code}
com.yahoo.ycsb.DBException:
org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed
54 actions: RegionTooBusyException: 54 times,
at com.yahoo.ycsb.db.HBaseClient.cleanup(HBaseClient.java:245)
at com.yahoo.ycsb.DBWrapper.cleanup(DBWrapper.java:73)
at com.yahoo.ycsb.ClientThread.run(Client.java:307)
Caused by:
org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed
54 actions: RegionTooBusyException: 54 times,
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:187)
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$500(AsyncProcess.java:171)
at
org.apache.hadoop.hbase.client.AsyncProcess.getErrors(AsyncProcess.java:897)
at
org.apache.hadoop.hbase.client.HTable.backgroundFlushCommits(HTable.java:961)
at
org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1225)
at com.yahoo.ycsb.db.HBaseClient.cleanup(HBaseClient.java:232)
... 2 more
{code}
On one of the RS
{code}
2014-02-11 08:45:58,714 INFO [regionserver60020.logRoller] wal.FSHLog: Too
many hlogs: logs=38, maxlogs=32; forcing flush of 23 regions(s):
97d8ae2f78910cc5ded5fbb1ddad8492, d396b8a1da05c871edcb68a15608fdf2,
01a68742a1be3a9705d574ad68fec1d7, 1250381046301e7465b6cf398759378e,
127c133f47d0419bd5ab66675aff76d4, 9f01c5d25ddc6675f750968873721253,
29c055b5690839c2fa357cd8e871741e, ca4e33e3eb0d5f8314ff9a870fc43463,
acfc6ae756e193b58d956cb71ccf0aa3, 187ea304069bc2a3c825bc10a59c7e84,
0ea411edc32d5c924d04bf126fa52d1e, e2f9331fc7208b1b230a24045f3c869e,
d9309ca864055eddf766a330352efc7a, 1a71bdf457288d449050141b5ff00c69,
0ba9089db28e977f86a27f90bbab9717, fdbb3242d3b673bbe4790a47bc30576f,
bbadaa1f0e62d8a8650080b824187850, b1a5de30d8603bd5d9022e09c574501b,
cc6a9fabe44347ed65e7c325faa72030, 313b17dbff2497f5041b57fe13fa651e,
6b788c498503ddd3e1433a4cd3fb4e39, 3d71274fe4f815882e9626e1cfa050d1,
acc43e4b42c1a041078774f4f20a3ff5
..
2014-02-11 08:47:49,580 INFO [regionserver60020.logRoller] wal.FSHLog: Too
many hlogs: logs=53, maxlogs=32; forcing flush of 2 regions(s):
fdbb3242d3b673bbe4790a47bc30576f, 6b788c498503ddd3e1433a4cd3fb4e39
{code}
{code}
2014-02-11 09:42:44,237 INFO [regionserver60020.periodicFlusher]
regionserver.HRegionServer: regionserver60020.periodicFlusher requesting
flush for region
usertable,user3654,1392107806977.fdbb3242d3b673bbe4790a47bc30576f. after a
delay of 16689
2014-02-11 09:42:44,237 INFO [regionserver60020.periodicFlusher]
regionserver.HRegionServer: regionserver60020.periodicFlusher requesting
flush for region
usertable,user6264,1392107806983.6b788c498503ddd3e1433a4cd3fb4e39. after a
delay of 15868
2014-02-11 09:42:54,238 INFO [regionserver60020.periodicFlusher]
regionserver.HRegionServer: regionserver60020.periodicFlusher requesting
flush for region
usertable,user3654,1392107806977.fdbb3242d3b673bbe4790a47bc30576f. after a
delay of 20847
2014-02-11 09:42:54,238 INFO [regionserver60020.periodicFlusher]
regionserver.HRegionServer: regionserver60020.periodicFlusher requesting
flush for region
usertable,user6264,1392107806983.6b788c498503ddd3e1433a4cd3fb4e39. after a
delay of 20099
2014-02-11 09:43:04,238 INFO
[jira] [Updated] (HBASE-11164) Document and test rolling updates from 0.98 - 1.0
[ https://issues.apache.org/jira/browse/HBASE-11164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11164: -- Fix Version/s: (was: 0.99.1) 0.99.2 Document and test rolling updates from 0.98 - 1.0 -- Key: HBASE-11164 URL: https://issues.apache.org/jira/browse/HBASE-11164 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Priority: Critical Fix For: 0.99.2 I think 1.0 should be rolling upgradable from 0.98 unless we break it intentionally for a specific reason. Unless there is such an issue, lets document that 1.0 and 0.98 should be rolling upgrade compatible. We should also test this before the 0.99 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-9738) Delete table and loadbalancer interference
[
https://issues.apache.org/jira/browse/HBASE-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-9738:
-
Fix Version/s: (was: 0.99.1)
0.99.2
Delete table and loadbalancer interference
--
Key: HBASE-9738
URL: https://issues.apache.org/jira/browse/HBASE-9738
Project: HBase
Issue Type: Bug
Reporter: Devaraj Das
Priority: Critical
Fix For: 0.99.2
I have noticed that when the balancer is computing a plan for region moves,
and a delete table is issued, there is some interference.
1. At time t1, user deleted the table.
2. This led to the master updating the meta table to remove the line for the
regioninfo for a region f2a9e2e9d70894c03f54ee5902bebee6.
{noformat}
2013-10-04 08:42:52,495 INFO [MASTER_TABLE_OPERATIONS-hor15n05:6-0]
catalog.MetaEditor: Deleted [{ENCODED = f2a9e2e9d70894c03f54ee5902bebee6,
NAME = 'usertable,,1380876170581.f2a9e2e9d70894c03f54ee5902bebee6.',
STARTKEY = '', ENDKEY = ''}]
{noformat}
3. However around the same time, the balancer kicked in, and reassigned the
region and made it online somewhere. It didn't check the fact (nor anyone
else did) that the table was indeed deleted.
{noformat}
2013-10-04 08:42:53,215 INFO
[hor15n05.gq1.ygridcore.net,6,1380869262259-BalancerChore]
master.HMaster: balance
hri=usertable,,1380876170581.f2a9e2e9d70894c03f54ee5902bebee6.,
src=hor15n09.gq1.ygridcore.net,60020,1380869263722,
dest=hor15n11.gq1.ygridcore.net,60020,1380869263682
{noformat}
.
{noformat}
2013-10-04 08:42:53,592 INFO [AM.ZK.Worker-pool2-t829] master.RegionStates:
Onlined f2a9e2e9d70894c03f54ee5902bebee6 on
hor15n11.gq1.ygridcore.net,60020,1380869263682
{noformat}
4. Henceforth, all the drop tables started giving warnings like
{noformat}
2013-10-04 08:45:17,587 INFO [RpcServer.handler=8,port=6]
master.HMaster: Client=hrt_qa//68.142.246.151 delete usertable
2013-10-04 08:45:17,631 DEBUG [RpcServer.handler=8,port=6]
lock.ZKInterProcessLockBase: Acquired a lock for
/hbase/table-lock/usertable/write-master:600
2013-10-04 08:45:17,637 WARN [RpcServer.handler=8,port=6]
catalog.MetaReader: No serialized HRegionInfo in
keyvalues={usertable,,1380876170581.f2a9e2e9d70894c03f54ee5902bebee6./info:seqnumDuringOpen/1380876173509/Put/vlen=8/mvcc=0,
usertable,,1380876170581.f2a9e2e9d70894c03f54ee5902bebee6./info:server/1380876173509/Put/vlen=32/mvcc=0,
usertable,,1380876170581.f2a9e2e9d70894c03f54ee5902bebee6./info:serverstartcode/1380876173509/Put/vlen=8/mvcc=0}
{noformat}
5. The create of the same table also fails since there is still state
(reincarnated, maybe) about the table in the master.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-9527) Review all old api that takes a table name as a byte array and ensure none can pass ns + tablename
[ https://issues.apache.org/jira/browse/HBASE-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-9527: - Fix Version/s: (was: 0.99.1) 0.99.2 Review all old api that takes a table name as a byte array and ensure none can pass ns + tablename -- Key: HBASE-9527 URL: https://issues.apache.org/jira/browse/HBASE-9527 Project: HBase Issue Type: Bug Reporter: stack Assignee: Sean Busbey Priority: Critical Fix For: 0.99.2 Go over all old APIs that take a table name and ensure that it is not possible to pass in a byte array that is a namespace + tablename; instead throw an exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11125) Introduce a higher level interface for registering interest in coprocessor upcalls
[ https://issues.apache.org/jira/browse/HBASE-11125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11125: -- Fix Version/s: (was: 0.99.1) 0.992 Introduce a higher level interface for registering interest in coprocessor upcalls -- Key: HBASE-11125 URL: https://issues.apache.org/jira/browse/HBASE-11125 Project: HBase Issue Type: New Feature Reporter: Andrew Purtell Priority: Critical Fix For: 0.992 We should introduce a higher level interface for managing the registration of 'user' code for execution from the low level hooks. It should not be necessary for coprocessor implementers to learn the universe of available low level hooks and the subtleties of their placement within HBase core code. Instead the higher level API should allow the implementer to describe their intent and then this API should choose the appropriate low level hook placement. A very desirable side effect is a layer of indirection between coprocessor implementers and the actual hooks. This will address the perennial complaint that the low level hooks change too much from release to release, as recently discussed during the RM panel at HBaseCon. If we try to avoid changing the particular placement and arguments of hook functions in response to those complaints, this can be an onerous constraint on necessary internals evolution. Instead we can direct coprocessor implementers to consider the new API and provide the same interface stability guarantees there as we do for client API, -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-9117) Remove HTablePool and all HConnection pooling related APIs
[ https://issues.apache.org/jira/browse/HBASE-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-9117: - Fix Version/s: (was: 0.99.1) 0.99.2 Remove HTablePool and all HConnection pooling related APIs -- Key: HBASE-9117 URL: https://issues.apache.org/jira/browse/HBASE-9117 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Assignee: Nick Dimiduk Priority: Critical Fix For: 0.99.2 Attachments: HBASE-9117.00.patch, HBASE-9117.01.patch, HBASE-9117.02.patch, HBASE-9117.03.patch, HBASE-9117.04.patch, HBASE-9117.05.patch, HBASE-9117.06.patch The recommended way is now: # Create an HConnection: HConnectionManager.createConnection(...) # Create a light HTable: HConnection.getTable(...) # table.close() # connection.close() All other API and pooling will be removed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9117) Remove HTablePool and all HConnection pooling related APIs
[ https://issues.apache.org/jira/browse/HBASE-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168460#comment-14168460 ] Enis Soztutar commented on HBASE-9117: -- The patch here was a bit more aggressive in replacing most of the managed connection usage which I guess turned out very hard to proceed the review process. Shall we break this into more bitable pieces? Some stuff has already went in around these. Should we close this and open smaller jiras? [~ndimiduk] wdyt? Remove HTablePool and all HConnection pooling related APIs -- Key: HBASE-9117 URL: https://issues.apache.org/jira/browse/HBASE-9117 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Assignee: Nick Dimiduk Priority: Critical Fix For: 0.99.2 Attachments: HBASE-9117.00.patch, HBASE-9117.01.patch, HBASE-9117.02.patch, HBASE-9117.03.patch, HBASE-9117.04.patch, HBASE-9117.05.patch, HBASE-9117.06.patch The recommended way is now: # Create an HConnection: HConnectionManager.createConnection(...) # Create a light HTable: HConnection.getTable(...) # table.close() # connection.close() All other API and pooling will be removed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-10295) Refactor the replication implementation to eliminate permanent zk node
[ https://issues.apache.org/jira/browse/HBASE-10295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10295: -- Fix Version/s: (was: 0.99.1) 0.99.2 Refactor the replication implementation to eliminate permanent zk node --- Key: HBASE-10295 URL: https://issues.apache.org/jira/browse/HBASE-10295 Project: HBase Issue Type: Bug Components: Replication Reporter: Honghua Feng Priority: Critical Labels: beginner Fix For: 0.99.2 Though this is a broader and bigger change, it original motivation derives from HBASE-8751: the newly introduced per-peer tableCFs attribute should be treated the same way as the peer-state, which is a permanent sub-node under peer node but using permanent zk node is deemed as an incorrect practice. So let's refactor to eliminate the permanent zk node. And the HBASE-8751 can then align its newly introduced per-peer tableCFs attribute with this *correct* implementation theme. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12083) Deprecate new HBaseAdmin() in favor of Connection.getAdmin()
[ https://issues.apache.org/jira/browse/HBASE-12083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-12083: -- Attachment: hbase-12083_v1.patch Here is a patch that does: - Make HTable and HBaseAdmin InterfaceAudience.Private - deprecate HBaseAdmin ctors - Document to use new APIs. - javadoc enhancements Deprecate new HBaseAdmin() in favor of Connection.getAdmin() Key: HBASE-12083 URL: https://issues.apache.org/jira/browse/HBASE-12083 Project: HBase Issue Type: Bug Reporter: Solomon Duskis Assignee: Solomon Duskis Priority: Critical Fix For: 2.0.0, 0.99.1 Attachments: hbase-12083_v1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12083) Deprecate new HBaseAdmin() in favor of Connection.getAdmin()
[ https://issues.apache.org/jira/browse/HBASE-12083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-12083: -- Status: Patch Available (was: Open) What do you guys think of this patch. Deprecate new HBaseAdmin() in favor of Connection.getAdmin() Key: HBASE-12083 URL: https://issues.apache.org/jira/browse/HBASE-12083 Project: HBase Issue Type: Bug Reporter: Solomon Duskis Assignee: Solomon Duskis Priority: Critical Fix For: 2.0.0, 0.99.1 Attachments: hbase-12083_v1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11852) Still see UnsupportedOperationException: CollectionUsage threshold is not supported
[
https://issues.apache.org/jira/browse/HBASE-11852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-11852:
--
Fix Version/s: (was: 0.99.1)
0.99.2
Still see UnsupportedOperationException: CollectionUsage threshold is not
supported
-
Key: HBASE-11852
URL: https://issues.apache.org/jira/browse/HBASE-11852
Project: HBase
Issue Type: Sub-task
Affects Versions: 0.99.0
Reporter: stack
Assignee: stack
Fix For: 2.0.0, 0.99.2
Trying to test tip of branch-1, I STILL see these exceptions in the log in
spite of the hacks the parent issue applied.
{code}
2014-08-28 10:48:28,659 DEBUG [295109315@qtp-1568239842-2]
jmx.JMXJsonServlet: getting attribute CollectionUsageThresholdCount of
java.lang:type=MemoryPool,name=Code Cache threw an exception
javax.management.RuntimeMBeanException:
java.lang.UnsupportedOperationException: CollectionUsage threshold is not
supported
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at
org.apache.hadoop.hbase.http.jmx.JMXJsonServlet.writeAttribute(JMXJsonServlet.java:333)
at
org.apache.hadoop.hbase.http.jmx.JMXJsonServlet.listBeans(JMXJsonServlet.java:311)
at
org.apache.hadoop.hbase.http.jmx.JMXJsonServlet.doGet(JMXJsonServlet.java:202)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at
org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1345)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.lang.UnsupportedOperationException: CollectionUsage threshold
is not supported
at
sun.management.MemoryPoolImpl.getCollectionUsageThresholdCount(MemoryPoolImpl.java:258)
at sun.reflect.GeneratedMethodAccessor43.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
at
[jira] [Updated] (HBASE-11927) If java7, use zip crc
[ https://issues.apache.org/jira/browse/HBASE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11927: -- Fix Version/s: (was: 0.99.1) 0.99.2 If java7, use zip crc - Key: HBASE-11927 URL: https://issues.apache.org/jira/browse/HBASE-11927 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.99.2 Attachments: c2021.crc2.svg, c2021.write.2.svg, c2021.zip.svg, crc32ct.svg Up in hadoop they have this change. Let me publish some graphs to show that it makes a difference (CRC is a massive amount of our CPU usage in my profiling of an upload because of compacting, flushing, etc.). We should also make use of native CRCings -- especially the 2.6 HDFS-6865 and ilk -- in hbase but that is another issue for now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11850) RpcClient can get stuck when closing
[ https://issues.apache.org/jira/browse/HBASE-11850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11850: -- Fix Version/s: (was: 0.99.1) 0.99.2 RpcClient can get stuck when closing Key: HBASE-11850 URL: https://issues.apache.org/jira/browse/HBASE-11850 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.99.0, 2.0.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 2.0.0, 0.99.2 we don't stop until the map in 'connections' is empty. the new connection is put in this map at creation, but if this connection is not used it will never be removed. No totally sure of the fix yet. Probability is low (but not zero. I saw it happening). The code is different in 0.98. It's 0.99+ issue only. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11314) Remove MultiVersionConsistencyControl#writeQueue
[ https://issues.apache.org/jira/browse/HBASE-11314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11314: -- Fix Version/s: (was: 0.99.1) 0.99.2 Remove MultiVersionConsistencyControl#writeQueue Key: HBASE-11314 URL: https://issues.apache.org/jira/browse/HBASE-11314 Project: HBase Issue Type: Improvement Reporter: Jeffrey Zhong Fix For: 0.99.2 This is a follow up improvement of HBase-8763. After we combined MVCC LogSeqId, the memstore read point advances in the logSeqId increasing order. The MultiVersionConsistencyControl#writeQueue can be possibly removed. In order to remove writeQueue, we need to return the disruptor sequence of the last WALEdit with durability = SYNC_WAL inside disruptor consumer code before assigning log sequence Id. If current WALEdit has durability = SYNC_WAL, we return the disruptor sequence of current WALEdit. During syncOrDefer, we always waits for the sequence of the last WALEdit has durability = SYNC_WAL. The tricky part is how to return the disruptor sequence of the last WALEdit with durability = SYNC_WAL cleanly from disruptor consumer. We could put those info in HLogKey because we always create a new HLogKey object for each change otherwise we have to create a new object for this purpose. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-10856) Prep for 1.0
[ https://issues.apache.org/jira/browse/HBASE-10856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10856: -- Fix Version/s: (was: 0.99.1) 0.99.2 Prep for 1.0 Key: HBASE-10856 URL: https://issues.apache.org/jira/browse/HBASE-10856 Project: HBase Issue Type: Umbrella Reporter: stack Fix For: 0.99.2 Tasks for 1.0 copied here from our '1.0.0' mailing list discussion. Idea is to file subtasks off this one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12083) Deprecate new HBaseAdmin() in favor of Connection.getAdmin()
[
https://issues.apache.org/jira/browse/HBASE-12083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168477#comment-14168477
]
Hadoop QA commented on HBASE-12083:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12674390/hbase-12083_v1.patch
against trunk revision .
ATTACHMENT ID: 12674390
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11320//console
This message is automatically generated.
Deprecate new HBaseAdmin() in favor of Connection.getAdmin()
Key: HBASE-12083
URL: https://issues.apache.org/jira/browse/HBASE-12083
Project: HBase
Issue Type: Bug
Reporter: Solomon Duskis
Assignee: Solomon Duskis
Priority: Critical
Fix For: 2.0.0, 0.99.1
Attachments: hbase-12083_v1.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-9206) namespace permissions
[ https://issues.apache.org/jira/browse/HBASE-9206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-9206: - Fix Version/s: (was: 0.99.1) 0.99.2 namespace permissions - Key: HBASE-9206 URL: https://issues.apache.org/jira/browse/HBASE-9206 Project: HBase Issue Type: Sub-task Reporter: Francis Liu Fix For: 0.99.2 Now that we have namespaces let's address how we can give admins more flexibility. Let's list out the privileges we'd like. Then we can map it to existing privileges and see if we need more. So far we have: 1. Modify namespace descriptor (ie quota, other values) 2. create namespace 3. delete namespace 4. list tables in namespace 5. create/drop tables in a namespace 6. All namespace's tables create 7. All namespace's tables write 8. All namespace's tables execute 9. All namespace's tables delete 10. All namespace's tables admin 1-3, is currently set to global admin only. Which seems acceptable to me. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12233) Bring back root table
[ https://issues.apache.org/jira/browse/HBASE-12233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168476#comment-14168476 ] Andrew Purtell commented on HBASE-12233: We could try this just with a 0.98 based feature branch and determine if it's worth pursuing further and elsewhere. Or do a 0.98 and master based feature branch both. I wouldn't mind refreshing the 0.98 one occasionally until we can make that determination. Bring back root table - Key: HBASE-12233 URL: https://issues.apache.org/jira/browse/HBASE-12233 Project: HBase Issue Type: Sub-task Reporter: Virag Kothari Assignee: Virag Kothari Attachments: HBASE-12233.patch First step towards splitting meta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12083) Deprecate new HBaseAdmin() in favor of Connection.getAdmin()
[ https://issues.apache.org/jira/browse/HBASE-12083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-12083: -- Attachment: hbase-12083_v2.patch rebased patch. Deprecate new HBaseAdmin() in favor of Connection.getAdmin() Key: HBASE-12083 URL: https://issues.apache.org/jira/browse/HBASE-12083 Project: HBase Issue Type: Bug Reporter: Solomon Duskis Assignee: Solomon Duskis Priority: Critical Fix For: 2.0.0, 0.99.1 Attachments: hbase-12083_v1.patch, hbase-12083_v2.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-8610) Introduce interfaces to support MultiWAL
[ https://issues.apache.org/jira/browse/HBASE-8610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-8610: - Fix Version/s: (was: 0.99.1) 0.99.2 Introduce interfaces to support MultiWAL Key: HBASE-8610 URL: https://issues.apache.org/jira/browse/HBASE-8610 Project: HBase Issue Type: Improvement Components: wal Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.2 Attachments: HBASE-8610_firstcut.patch As the heading says this JIRA is specific to adding interfaces to support MultiWAL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12174) MR support for cell TTLs
[ https://issues.apache.org/jira/browse/HBASE-12174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-12174: -- Fix Version/s: (was: 0.99.1) 0.99.2 MR support for cell TTLs Key: HBASE-12174 URL: https://issues.apache.org/jira/browse/HBASE-12174 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.8, 0.99.2 Consider supporting cell TTLs where we already have MR code that can add operation attributes, like the TSV importer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12170) TestReplicaWithCluster.testReplicaAndReplication timeouts
[
https://issues.apache.org/jira/browse/HBASE-12170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar resolved HBASE-12170.
---
Resolution: Fixed
Recent builds seems good for this test. Resolving. Thanks Stack.
TestReplicaWithCluster.testReplicaAndReplication timeouts
-
Key: HBASE-12170
URL: https://issues.apache.org/jira/browse/HBASE-12170
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Fix For: 2.0.0, 0.99.1
Attachments: to.txt
On our internal rig I got a timeout. Looking over the test, it looks to be
making progress if slow. The test does a crazy amount of stuff too... so let
me just up the timeout for now.
{code}
org.apache.hadoop.hbase.client.TestReplicaWithCluster.testReplicaAndReplication
Failing for the past 1 build (Since Unstable#10 )
Took 0.42 sec.
Error Message
test timed out after 12 milliseconds
Stacktrace
java.lang.Exception: test timed out after 12 milliseconds
at java.lang.Thread.sleep(Native Method)
at
org.apache.hadoop.hbase.client.HBaseAdmin.disableTable(HBaseAdmin.java:999)
at
org.apache.hadoop.hbase.client.TestReplicaWithCluster.testReplicaAndReplication(TestReplicaWithCluster.java:302)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-10560) Per cell TTLs
[ https://issues.apache.org/jira/browse/HBASE-10560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10560: -- Fix Version/s: (was: 0.99.1) 0.99.2 Per cell TTLs - Key: HBASE-10560 URL: https://issues.apache.org/jira/browse/HBASE-10560 Project: HBase Issue Type: New Feature Affects Versions: 0.98.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.8, 0.99.2 Now that we have cell tags, we can optionally store TTLs per cell. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12053) SecurityBulkLoadEndPoint set 777 permission on input data files
[
https://issues.apache.org/jira/browse/HBASE-12053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168500#comment-14168500
]
Enis Soztutar commented on HBASE-12053:
---
I'm +1 for this patch, but would be more comfortable if we can get another
review. cc [~andrew.purt...@gmail.com], [~toffer].
SecurityBulkLoadEndPoint set 777 permission on input data files
Key: HBASE-12053
URL: https://issues.apache.org/jira/browse/HBASE-12053
Project: HBase
Issue Type: Bug
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 2.0.0, 0.98.8, 0.99.1
Attachments: HBASE-12053.patch
We have code in SecureBulkLoadEndpoint#secureBulkLoadHFiles
{code}
LOG.trace(Setting permission for: + p);
fs.setPermission(p, PERM_ALL_ACCESS);
{code}
This is against the point we use staging folder for secure bulk load.
Currently we create a hidden staging folder which has ALL_ACCESS permission
and we use doAs to move input files into staging folder. Therefore, we
should not set 777 permission on the original input data files but files in
staging folder after move.
This may comprise security setting especially when there is an error we
move the file with 777 permission back.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12053) SecurityBulkLoadEndPoint set 777 permission on input data files
[
https://issues.apache.org/jira/browse/HBASE-12053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-12053:
--
Fix Version/s: (was: 0.99.1)
0.99.2
SecurityBulkLoadEndPoint set 777 permission on input data files
Key: HBASE-12053
URL: https://issues.apache.org/jira/browse/HBASE-12053
Project: HBase
Issue Type: Bug
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 2.0.0, 0.98.8, 0.99.2
Attachments: HBASE-12053.patch
We have code in SecureBulkLoadEndpoint#secureBulkLoadHFiles
{code}
LOG.trace(Setting permission for: + p);
fs.setPermission(p, PERM_ALL_ACCESS);
{code}
This is against the point we use staging folder for secure bulk load.
Currently we create a hidden staging folder which has ALL_ACCESS permission
and we use doAs to move input files into staging folder. Therefore, we
should not set 777 permission on the original input data files but files in
staging folder after move.
This may comprise security setting especially when there is an error we
move the file with 777 permission back.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-11179) API parity between mapred and mapreduce
[ https://issues.apache.org/jira/browse/HBASE-11179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11179: -- Labels: beginner (was: ) API parity between mapred and mapreduce --- Key: HBASE-11179 URL: https://issues.apache.org/jira/browse/HBASE-11179 Project: HBase Issue Type: Sub-task Components: mapreduce Reporter: Nick Dimiduk Labels: beginner Fix For: 0.99.2 This ticket is for bringing the mapred package up to feature parity with mapreduce. Might become an umbrella ticket in and of itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11179) API parity between mapred and mapreduce
[ https://issues.apache.org/jira/browse/HBASE-11179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11179: -- Fix Version/s: (was: 0.99.1) 0.99.2 API parity between mapred and mapreduce --- Key: HBASE-11179 URL: https://issues.apache.org/jira/browse/HBASE-11179 Project: HBase Issue Type: Sub-task Components: mapreduce Reporter: Nick Dimiduk Labels: beginner Fix For: 0.99.2 This ticket is for bringing the mapred package up to feature parity with mapreduce. Might become an umbrella ticket in and of itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6581) Build with hadoop.profile=3.0
[ https://issues.apache.org/jira/browse/HBASE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-6581: - Fix Version/s: (was: 0.99.1) 0.99.2 Build with hadoop.profile=3.0 - Key: HBASE-6581 URL: https://issues.apache.org/jira/browse/HBASE-6581 Project: HBase Issue Type: Bug Reporter: Eric Charles Assignee: Eric Charles Fix For: 2.0.0, 0.99.2 Attachments: 6581.txt, 6581v2.txt, HBASE-6581-1.patch, HBASE-6581-2.patch, HBASE-6581-20130821.patch, HBASE-6581-3.patch, HBASE-6581-4.patch, HBASE-6581-5.patch, HBASE-6581-6.patch, HBASE-6581-7.patch, HBASE-6581-8-pom.patch, HBASE-6581.diff, HBASE-6581.diff Building trunk with hadoop.profile=3.0 gives exceptions (see [1]) due to change in the hadoop maven modules naming (and also usage of 3.0-SNAPSHOT instead of 3.0.0-SNAPSHOT in hbase-common). I can provide a patch that would move most of hadoop dependencies in their respective profiles and will define the correct hadoop deps in the 3.0 profile. Please tell me if that's ok to go this way. Thx, Eric [1] $ mvn clean install -Dhadoop.profile=3.0 [INFO] Scanning for projects... [ERROR] The build could not read 3 projects - [Help 1] [ERROR] [ERROR] The project org.apache.hbase:hbase-server:0.95-SNAPSHOT (/d/hbase.svn/hbase-server/pom.xml) has 3 errors [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 655, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 659, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 663, column 21 [ERROR] [ERROR] The project org.apache.hbase:hbase-common:0.95-SNAPSHOT (/d/hbase.svn/hbase-common/pom.xml) has 3 errors [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 170, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 174, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 178, column 21 [ERROR] [ERROR] The project org.apache.hbase:hbase-it:0.95-SNAPSHOT (/d/hbase.svn/hbase-it/pom.xml) has 3 errors [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 220, column 18 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 224, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 228, column 21 [ERROR] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6970) hbase-deamon.sh creates/updates pid file even when that start failed.
[
https://issues.apache.org/jira/browse/HBASE-6970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-6970:
-
Fix Version/s: (was: 0.99.1)
2.0.0
hbase-deamon.sh creates/updates pid file even when that start failed.
-
Key: HBASE-6970
URL: https://issues.apache.org/jira/browse/HBASE-6970
Project: HBase
Issue Type: Bug
Components: Usability
Reporter: Lars Hofhansl
Fix For: 2.0.0
We just ran into a strange issue where could neither start nor stop services
with hbase-deamon.sh.
The problem is this:
{code}
nohup nice -n $HBASE_NICENESS $HBASE_HOME/bin/hbase \
--config ${HBASE_CONF_DIR} \
$command $@ $startStop $logout 21 /dev/null
echo $! $pid
{code}
So the pid file is created or updated even when the start of the service
failed. The next stop command will then fail, because the pid file has the
wrong pid in it.
Edit: Spelling and more spelling errors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
