[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619182#comment-13619182 ] Andrew Purtell commented on HBASE-7904: --- bq. 0.95 should not ship with 2.0.4-SNAPSHOT build. That was why I left this open. I think it is confusing, because the patch on this issue has already been committed to the target branches. Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Priority: Critical Fix For: 0.95.0, 0.98.0 Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619186#comment-13619186 ] Andrew Purtell commented on HBASE-7904: --- bq. I exchanged a few emails with Nicolas who was the author of CompoundConfiguration. All interactions should be on JIRA or on the dev@ or user@ email lists. Decisions made in private communication are against the Apache Way. Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Priority: Critical Fix For: 0.95.0, 0.98.0 Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8174) Backport HBASE-8161(setting blocking file count on table level doesn't work) to 0.94
[ https://issues.apache.org/jira/browse/HBASE-8174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619192#comment-13619192 ] Hudson commented on HBASE-8174: --- Integrated in HBase-0.94 #934 (See [https://builds.apache.org/job/HBase-0.94/934/]) HBASE-8174 Backport HBASE-8161(setting blocking file count on table level doesn't work) to 0.94 (clockfly) (Revision 1463204) Result = SUCCESS tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java Backport HBASE-8161(setting blocking file count on table level doesn't work) to 0.94 Key: HBASE-8174 URL: https://issues.apache.org/jira/browse/HBASE-8174 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.94.5 Reporter: clockfly Assignee: clockfly Priority: Minor Fix For: 0.94.7 Attachments: hbase-8174.patch, HBASE-8174.patch.0.94.v1 Currently, the blocking file count hbase.hstore.blockingStoreFiles is configured at region server level. We should allow it to be configured at Table level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8213) global authorization may lose efficacy
[
https://issues.apache.org/jira/browse/HBASE-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619193#comment-13619193
]
Hudson commented on HBASE-8213:
---
Integrated in HBase-0.94 #934 (See
[https://builds.apache.org/job/HBase-0.94/934/])
HBASE-8213. Global authorization may lose efficacy (Revision 1463197)
Result = SUCCESS
apurtell :
Files :
*
/hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java
*
/hbase/branches/0.94/security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
global authorization may lose efficacy
---
Key: HBASE-8213
URL: https://issues.apache.org/jira/browse/HBASE-8213
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.95.0, 0.96.0, 0.94.6
Reporter: Jieshan Bean
Assignee: Jieshan Bean
Priority: Critical
Fix For: 0.95.0, 0.98.0, 0.94.7
Attachments: HBASE-8213-94.patch, HBASE-8213-trunk.patch
It depends on the order of which region be opened first.
Suppose we have one 1 regionserver and only 1 user region REGION-A on this
server, _acl_ region was on another regionserver. _acl_ was opened a few
seconds before REGION-A.
The global authorization data read from Zookeeper was overwritten by the data
read from configuration.
{code}
private TableAuthManager(ZooKeeperWatcher watcher, Configuration conf)
throws IOException {
this.conf = conf;
this.zkperms = new ZKPermissionWatcher(watcher, this, conf);
try {
// Read global authorization data from zookeeper.
this.zkperms.start();
} catch (KeeperException ke) {
LOG.error(ZooKeeper initialization failed, ke);
}
// It will overwrite globalCache.
// initialize global permissions based on configuration
globalCache = initGlobal(conf);
}
{code}
This issue can be easily reproduced by below steps:
1. Start a cluster with 3 regionservers.
2. Create a new table T1.
3. grant a new user USER-A with global authorization.
4. Kill 1 regionserver RS3 and switch balance off.
5. Start regionserver RS3.
6. Assign region T1 to RS3.
7. Put data with user USER-A.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6881) All regionservers are marked offline even there is still one up
[
https://issues.apache.org/jira/browse/HBASE-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619194#comment-13619194
]
Hudson commented on HBASE-6881:
---
Integrated in HBase-0.94 #934 (See
[https://builds.apache.org/job/HBase-0.94/934/])
HBASE-7925 Back port HBASE-6881 into 0.94 (Rajeshbabu) (Revision 1463155)
Result = SUCCESS
tedyu :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
All regionservers are marked offline even there is still one up
---
Key: HBASE-6881
URL: https://issues.apache.org/jira/browse/HBASE-6881
Project: HBase
Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Fix For: 0.95.0
Attachments: trunk-6881.patch, trunk-6881_v3.patch
{noformat}
+RegionPlan newPlan = plan;
+if (!regionAlreadyInTransitionException) {
+ // Force a new plan and reassign. Will return null if no servers.
+ newPlan = getRegionPlan(state, plan.getDestination(), true);
+}
+if (newPlan == null) {
this.timeoutMonitor.setAllRegionServersOffline(true);
LOG.warn(Unable to find a viable location to assign region +
state.getRegion().getRegionNameAsString());
{noformat}
Here, when newPlan is null, plan.getDestination() could be up actually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8161) setting blocking file count on table level doesn't work
[ https://issues.apache.org/jira/browse/HBASE-8161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619195#comment-13619195 ] Hudson commented on HBASE-8161: --- Integrated in HBase-0.94 #934 (See [https://builds.apache.org/job/HBase-0.94/934/]) HBASE-8174 Backport HBASE-8161(setting blocking file count on table level doesn't work) to 0.94 (clockfly) (Revision 1463204) Result = SUCCESS tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java setting blocking file count on table level doesn't work --- Key: HBASE-8161 URL: https://issues.apache.org/jira/browse/HBASE-8161 Project: HBase Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Fix For: 0.95.0, 0.98.0 Attachments: HBASE-8161-v0-hmm.patch, HBASE-8161-v0.patch MemStoreFlusher always uses server-wide config... it should ask the store correct question instead -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7925) Back port HBASE-6881 into 0.94
[ https://issues.apache.org/jira/browse/HBASE-7925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619196#comment-13619196 ] Hudson commented on HBASE-7925: --- Integrated in HBase-0.94 #934 (See [https://builds.apache.org/job/HBase-0.94/934/]) HBASE-7925 Back port HBASE-6881 into 0.94 (Rajeshbabu) (Revision 1463155) Result = SUCCESS tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java Back port HBASE-6881 into 0.94 -- Key: HBASE-7925 URL: https://issues.apache.org/jira/browse/HBASE-7925 Project: HBase Issue Type: Bug Reporter: rajeshbabu Assignee: rajeshbabu Fix For: 0.94.7 Attachments: HBASE-7925_94.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8225) [replication] minor code bug when registering ReplicationLogCleaner
[
https://issues.apache.org/jira/browse/HBASE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619218#comment-13619218
]
Lars Hofhansl commented on HBASE-8225:
--
+1 for 0.94.
[replication] minor code bug when registering ReplicationLogCleaner
---
Key: HBASE-8225
URL: https://issues.apache.org/jira/browse/HBASE-8225
Project: HBase
Issue Type: Bug
Components: Replication
Affects Versions: 0.94.6
Reporter: Jerry He
Assignee: Jerry He
Priority: Minor
Fix For: 0.95.0, 0.96.0, 0.94.7
Attachments: HBASE-8255-0.94.patch, HBASE-8255-trunk.patch
We try to register ReplicationLogCleaner by default. This is done by calling
Replication.decorateMasterConfiguration()in the master.
In the current Replication.decorateMasterConfiguration():
...
String plugins = conf.get(HBASE_MASTER_LOGCLEANER_PLUGINS);
if (!plugins.contains(ReplicationLogCleaner.class.toString())) {
conf.set(HBASE_MASTER_LOGCLEANER_PLUGINS,
plugins + , + ReplicationLogCleaner.class.getCanonicalName());
}
...
ReplicationLogCleaner.class.toString() will return prefix 'class' to the full
class name, which will make the if checking mostly useless.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7670) Synchronized operation in CatalogTracker would block handling ZK Event for long time
[
https://issues.apache.org/jira/browse/HBASE-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619221#comment-13619221
]
Sergey Shelukhin commented on HBASE-7670:
-
Is this issue relevant anymore given the recent root/meta changes?
Synchronized operation in CatalogTracker would block handling ZK Event for
long time
Key: HBASE-7670
URL: https://issues.apache.org/jira/browse/HBASE-7670
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.4
Reporter: chunhui shen
Assignee: chunhui shen
Priority: Critical
Fix For: 0.95.0
Attachments: HBASE-7670.patch, HBASE-7670.patch
We found ZK event not be watched by master for a long time in our testing.
It seems one ZK-Event-Handle thread block it.
Attaching some logs on master
{code}
2013-01-16 22:18:55,667 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_OPENED,
2013-01-16 22:18:56,270 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_OPENED,
...
2013-01-16 23:55:33,259 INFO org.apache.hadoop.hbase.catalog.CatalogTracker:
Retrying
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
attempts=100, exceptions:
at
org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:183)
at org.apache.hadoop.hbase.client.HTable.get(HTable.java:676)
at org.apache.hadoop.hbase.catalog.MetaReader.get(MetaReader.java:247)
at
org.apache.hadoop.hbase.catalog.MetaReader.getRegion(MetaReader.java:349)
at
org.apache.hadoop.hbase.catalog.MetaReader.readRegionLocation(MetaReader.java:289)
at
org.apache.hadoop.hbase.catalog.MetaReader.getMetaRegionLocation(MetaReader.java:276)
at
org.apache.hadoop.hbase.catalog.CatalogTracker.getMetaServerConnection(CatalogTracker.java:424)
at
org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:489)
at
org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:451)
at
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:289)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2013-01-16 23:55:33,261 WARN
org.apache.hadoop.hbase.master.AssignmentManager: Attempted to handle region
transition for server but server is not online
{code}
Between 2013-01-16 22:18:56 and 2013-01-16 23:55:33, there is no any logs
about handling ZK Event.
{code}
this.metaNodeTracker = new MetaNodeTracker(zookeeper, throwableAborter) {
public void nodeDeleted(String path) {
if (!path.equals(node)) return;
ct.resetMetaLocation();
}
}
public void resetMetaLocation() {
LOG.debug(Current cached META location, + metaLocation +
, is not valid, resetting);
synchronized(this.metaAvailable) {
this.metaAvailable.set(false);
this.metaAvailable.notifyAll();
}
}
private AdminProtocol getMetaServerConnection(){
synchronized (metaAvailable){
...
ServerName newLocation = MetaReader.getMetaRegionLocation(this);
...
}
}
{code}
From the above code, we would found that nodeDeleted() would wait
synchronized (metaAvailable) until MetaReader.getMetaRegionLocation(this)
done,
however, getMetaRegionLocation() could be retrying for a long time
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8213) global authorization may lose efficacy
[
https://issues.apache.org/jira/browse/HBASE-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619225#comment-13619225
]
Hudson commented on HBASE-8213:
---
Integrated in HBase-TRUNK #4004 (See
[https://builds.apache.org/job/HBase-TRUNK/4004/])
HBASE-8213. Global authorization may lose efficacy (Revision 1463189)
Result = FAILURE
apurtell :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
global authorization may lose efficacy
---
Key: HBASE-8213
URL: https://issues.apache.org/jira/browse/HBASE-8213
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.95.0, 0.96.0, 0.94.6
Reporter: Jieshan Bean
Assignee: Jieshan Bean
Priority: Critical
Fix For: 0.95.0, 0.98.0, 0.94.7
Attachments: HBASE-8213-94.patch, HBASE-8213-trunk.patch
It depends on the order of which region be opened first.
Suppose we have one 1 regionserver and only 1 user region REGION-A on this
server, _acl_ region was on another regionserver. _acl_ was opened a few
seconds before REGION-A.
The global authorization data read from Zookeeper was overwritten by the data
read from configuration.
{code}
private TableAuthManager(ZooKeeperWatcher watcher, Configuration conf)
throws IOException {
this.conf = conf;
this.zkperms = new ZKPermissionWatcher(watcher, this, conf);
try {
// Read global authorization data from zookeeper.
this.zkperms.start();
} catch (KeeperException ke) {
LOG.error(ZooKeeper initialization failed, ke);
}
// It will overwrite globalCache.
// initialize global permissions based on configuration
globalCache = initGlobal(conf);
}
{code}
This issue can be easily reproduced by below steps:
1. Start a cluster with 3 regionservers.
2. Create a new table T1.
3. grant a new user USER-A with global authorization.
4. Kill 1 regionserver RS3 and switch balance off.
5. Start regionserver RS3.
6. Assign region T1 to RS3.
7. Put data with user USER-A.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8164) TestTableLockManager fails intermittently in trunk builds
[ https://issues.apache.org/jira/browse/HBASE-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619226#comment-13619226 ] Hudson commented on HBASE-8164: --- Integrated in HBase-TRUNK #4004 (See [https://builds.apache.org/job/HBase-TRUNK/4004/]) HBASE-8164 TestTableLockManager fails intermittently in trunk builds (Revision 1463194) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestTableLockManager.java TestTableLockManager fails intermittently in trunk builds - Key: HBASE-8164 URL: https://issues.apache.org/jira/browse/HBASE-8164 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.95.0, 0.98.0 Attachments: 8164-addendum-2.txt, 8164-addendum.txt, 8164-v2.txt, 8164-v3.txt, 8164-v4.txt, HBASE-8164_addendum_3.patch, HBASE-8164_addendum_4.patch, HBASE-8164_addendum_5.patch In build #3979: testTableReadLock(org.apache.hadoop.hbase.master.TestTableLockManager): test timed out after 60 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8236) Set finalName property in hbase-assembly else basename is hbase-assembly rather than hbase.
[ https://issues.apache.org/jira/browse/HBASE-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619227#comment-13619227 ] Hudson commented on HBASE-8236: --- Integrated in HBase-TRUNK #4004 (See [https://builds.apache.org/job/HBase-TRUNK/4004/]) HBASE-8236 Set finalName property in hbase-assembly else basename is hbase-assembly rather than hbase (Revision 1463258) Result = FAILURE stack : Files : * /hbase/trunk/hbase-assembly/pom.xml Set finalName property in hbase-assembly else basename is hbase-assembly rather than hbase. --- Key: HBASE-8236 URL: https://issues.apache.org/jira/browse/HBASE-8236 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.95.0 Attachments: 8236.txt, version.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619229#comment-13619229 ] stack commented on HBASE-7904: -- bq. Do you happen to know what this incompatibility was (and against which version) ? [~sseth] No. This issue started out as an incompability issue (see original [~ted_yu] comments at head of issue) and then ended w/ some configuration manipulations that seem gratiuitous. I ask the author what fixed the issue and I get that they think the config. manipulations are all that is necessary. I have spent a bunch of time trying to follow what is going on here, but gave up, and +1'd it as a means of putting the issue behind us since the patch seemed relatively harmless. bq. As an example, can TestTableMapReduce run in parallel with TestHFileOutputFormat ?, or can multiple tests within TestTableMapReduce run in parallel, or something else. Yes, hbase tests can be run in parallel: e.g. TestTableMapReduce and TestHFileOutputFormat run at same time. Our surefire runs our test suite by firing off N JVMs each running a distinct, long-running test. What we do not do is set up a mini mr cluster and then run many unit tests against. bq. I exchanged a few emails with Nicolas who was the author of CompoundConfiguration. [~ted_yu] And you say nothing in here about it? Why do it in private? They call it open source for a reason. Suggest we revert this patch. Seems broke that CPs see another config. because Chane to HRegion to not use a 'CompoundConfiguration' when talking to HDFS, due to the inability to change the RPC engine with CompoundConfiguration. (Causes several *SecureLoad tests to hang) according to https://issues.apache.org/jira/browse/YARN-449 (a conclusion that is absent here). Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Priority: Critical Fix For: 0.95.0, 0.98.0 Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[
https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619234#comment-13619234
]
Ted Yu commented on HBASE-7904:
---
I tried the following change:
{code}
Index:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
===
---
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
(revision 1463235)
+++
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
(working copy)
@@ -484,7 +484,7 @@
this.rsAccounting = this.rsServices.getRegionServerAccounting();
// don't initialize coprocessors if not running within a regionserver
// TODO: revisit if coprocessors should load in other cases
- this.coprocessorHost = new RegionCoprocessorHost(this, rsServices,
baseConf);
+ this.coprocessorHost = new RegionCoprocessorHost(this, rsServices, conf);
this.metricsRegionWrapper = new MetricsRegionWrapperImpl(this);
this.metricsRegion = new MetricsRegion(this.metricsRegionWrapper);
} else {
Index:
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
===
---
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
(revision 1463235)
+++
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
(working copy)
@@ -121,7 +121,7 @@
baseStagingDir = SecureBulkLoadUtil.getBaseStagingDir(conf);
try {
- fs = FileSystem.get(conf);
+ fs = FileSystem.get(new Configuration(conf));
fs.mkdirs(baseStagingDir, PERM_HIDDEN);
fs.setPermission(baseStagingDir, PERM_HIDDEN);
//no sticky bit in hadoop-1.0, making directory nonempty so it never
gets erased
{code}
But I still got:
{code}
2013-04-01 14:43:38,337 ERROR [IPC Server handler 0 on 52052]
access.SecureBulkLoadEndpoint$1(240): Failed to complete bulk load
java.lang.UnsupportedOperationException: Immutable Configuration
at
org.apache.hadoop.hbase.CompoundConfiguration.setClass(CompoundConfiguration.java:474)
at org.apache.hadoop.ipc.RPC.setProtocolEngine(RPC.java:193)
at
org.apache.hadoop.hdfs.NameNodeProxies.createNNProxyWithClientProtocol(NameNodeProxies.java:249)
at
org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:168)
at
org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:129)
at org.apache.hadoop.hdfs.DFSClient.init(DFSClient.java:421)
at org.apache.hadoop.hdfs.DFSClient.init(DFSClient.java:388)
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:126)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2312)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:87)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2346)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2328)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:352)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:164)
at
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint$1.run(SecureBulkLoadEndpoint.java:224)
at
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint$1.run(SecureBulkLoadEndpoint.java:218)
{code}
I also tried this:
{code}
Index:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
===
---
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
(revision 1463235)
+++
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
(working copy)
@@ -484,7 +484,7 @@
this.rsAccounting = this.rsServices.getRegionServerAccounting();
// don't initialize coprocessors if not running within a regionserver
// TODO: revisit if coprocessors should load in other cases
- this.coprocessorHost = new RegionCoprocessorHost(this, rsServices,
baseConf);
+ this.coprocessorHost = new RegionCoprocessorHost(this, rsServices, conf);
this.metricsRegionWrapper = new MetricsRegionWrapperImpl(this);
this.metricsRegion = new MetricsRegion(this.metricsRegionWrapper);
} else {
Index:
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
===
---
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
(revision 1463235)
+++
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
(working copy)
@@ -31,6 +31,7 @@
import org.apache.hadoop.fs.permission.FsPermission;
import org.apache.hadoop.hbase.Coprocessor;
[jira] [Commented] (HBASE-8164) TestTableLockManager fails intermittently in trunk builds
[ https://issues.apache.org/jira/browse/HBASE-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619235#comment-13619235 ] Enis Soztutar commented on HBASE-8164: -- Blessed be the unit test fixers! bq. Sorry guys for writing a seemingly unfixable unit test :) TestTableLockManager fails intermittently in trunk builds - Key: HBASE-8164 URL: https://issues.apache.org/jira/browse/HBASE-8164 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.95.0, 0.98.0 Attachments: 8164-addendum-2.txt, 8164-addendum.txt, 8164-v2.txt, 8164-v3.txt, 8164-v4.txt, HBASE-8164_addendum_3.patch, HBASE-8164_addendum_4.patch, HBASE-8164_addendum_5.patch In build #3979: testTableReadLock(org.apache.hadoop.hbase.master.TestTableLockManager): test timed out after 60 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8238) xref docs missing from site gen
stack created HBASE-8238: Summary: xref docs missing from site gen Key: HBASE-8238 URL: https://issues.apache.org/jira/browse/HBASE-8238 Project: HBase Issue Type: Bug Reporter: stack -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8238) xref docs missing from site gen
[ https://issues.apache.org/jira/browse/HBASE-8238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8238: - Attachment: 8238.txt Small fix xref docs missing from site gen --- Key: HBASE-8238 URL: https://issues.apache.org/jira/browse/HBASE-8238 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8238.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619238#comment-13619238 ] Andrew Purtell commented on HBASE-7904: --- bq. Suggest we revert this patch. Seems broke that CPs see another config. +1 for revert Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Priority: Critical Fix For: 0.95.0, 0.98.0 Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8238) xref docs missing from site gen
[ https://issues.apache.org/jira/browse/HBASE-8238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-8238. -- Resolution: Fixed Fix Version/s: 0.95.0 Assignee: stack Committed to 0.95 and trunk. xref docs missing from site gen --- Key: HBASE-8238 URL: https://issues.apache.org/jira/browse/HBASE-8238 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.95.0 Attachments: 8238.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619240#comment-13619240 ] Ted Yu commented on HBASE-7904: --- bq. Suggest we revert this patch I agree. We should consider a better design first. What about the notification that checkin to 0.95 is on hold ? Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Priority: Critical Fix For: 0.95.0, 0.98.0 Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[
https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619246#comment-13619246
]
Ted Yu commented on HBASE-7904:
---
Another solution I can think of is to expose two Configurations from
CoprocessorEnvironment: one that is immutable, the other mutable. The second
one would be used for this call:
{code}
fs = FileSystem.get(conf);
{code}
BTW, here is javadoc for FileSystem.get():
{code}
* Get a filesystem instance based on the uri, the passed
* configuration and the user
* @param uri of the filesystem
* @param conf the configuration to use
{code}
I don't see mentioning that conf cannot be immutable.
Make mapreduce jobs pass based on 2.0.4-alpha
-
Key: HBASE-7904
URL: https://issues.apache.org/jira/browse/HBASE-7904
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Critical
Fix For: 0.95.0, 0.98.0
Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt,
7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt,
7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt,
7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt,
7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt
2.0.3-alpha has been released.
We should upgrade the dependency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619251#comment-13619251 ] stack commented on HBASE-7904: -- [~ted_yu] Please back this out of trunk and 0.95 (I tried but cannot figure what has been committed w/o doing a bunch of work) Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Priority: Critical Fix For: 0.95.0, 0.98.0 Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619252#comment-13619252 ] Andrew Purtell commented on HBASE-7904: --- bq. Another solution I can think of is to expose two Configurations from CoprocessorEnvironment: one that is immutable, the other mutable. The second one would be used for this call I think that's closer to a solution. I think what you want is to take the immutable CompoundConfiguration and create a new mutable Configuration with it to pass to the RegionCoprocessorHost constructor, one initialized with the overrides provided by the compound view. But I suggest reverting this patch and tackling the issues here with focused JIRAs. Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Priority: Critical Fix For: 0.95.0, 0.98.0 Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8224) Add '-hadoop1' or '-hadoop2' to our version string
[ https://issues.apache.org/jira/browse/HBASE-8224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619255#comment-13619255 ] stack commented on HBASE-8224: -- [~enis] How would that work? What would the artifact be? Would we have two poms or would be branch to do hadoop2 builds making artifact name change in branch? Thanks. Add '-hadoop1' or '-hadoop2' to our version string -- Key: HBASE-8224 URL: https://issues.apache.org/jira/browse/HBASE-8224 Project: HBase Issue Type: Task Reporter: stack Attachments: 8224-adding.classifiers.txt So we can publish both the hadoop1 and the hadoop2 jars to a maven repository, and so we can publish two packages, one for hadoop1 and one for hadoop2, given how maven works, our only alternative (to the best of my knowledge and after consulting others) is by amending the version string to include hadoop1 or hadoop2. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7410) [snapshots] add snapshot/clone/restore/export docs to ref guide
[ https://issues.apache.org/jira/browse/HBASE-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619258#comment-13619258 ] Sergey Shelukhin commented on HBASE-7410: - First paragraph initially says that the only ways yto do X are these, then discusses snapshots. Maybe it should qualify the previous methods differently, like as the previously discussed. I think it would be even better if the first paragraph could start with what snapshots allow you to do and only mention the disadvantages of the other stuff after that. bq. the the region server dup bq. HBase Snapshots allow you to take a snapshot of the table without too much impact and cloning/restoring *a* table, and clone/restore? bq. set to true the varnamehbase.snapshot.enabled/varname property. set the ... to true bq. that is enable or disabled. enabled; or regardless of whether it is enabled or disabled. bq. involve any data-copy operation any data copying? bq. but after a compaction, or table deletion the files needed by the snapshot are moved to an archive directory instead of being deleted. Is user supposed to know what this means? I personally have hard time understanding the implications :) bq. doesn't involve data copies data copying? bq. restore operation require requires bq. snapshot state snapshotted state/the state at the time when the snapshot was taken? bq. stop replication and redo the bootstra This sentence doesn't appear to be fi :) bq. The ExportSnapshot tool allows to copy all the data related to a snapshot (hfiles, logs, snapshot metadata) and copy it to another cluster. allows one or remove allows and just say copies? Double copy. Also, from admin's perspective, should it just say that the tool copies entire snapshot to another cluster, and not enumerate the files? bq. The tool execute executes bq. and since it works at file-system level the hbase cluster can be offline. Maybe does not have to be online to avoid potential confusion would be better. [snapshots] add snapshot/clone/restore/export docs to ref guide --- Key: HBASE-7410 URL: https://issues.apache.org/jira/browse/HBASE-7410 Project: HBase Issue Type: Improvement Components: documentation, snapshots Affects Versions: hbase-6055 Reporter: Jonathan Hsieh Priority: Blocker Fix For: 0.95.0 Attachments: HBASE-7410-v0.patch This will include additions to the ref guide about the different operations provided and how to use them. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619259#comment-13619259 ] Ted Yu commented on HBASE-7904: --- Backed out of 0.95 and trunk. Will attach patch v10 soon. Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Priority: Critical Fix For: 0.95.0, 0.98.0 Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5923) Cleanup checkAndXXX logic
[
https://issues.apache.org/jira/browse/HBASE-5923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619260#comment-13619260
]
Sergey Shelukhin commented on HBASE-5923:
-
hmm...
Cleanup checkAndXXX logic
-
Key: HBASE-5923
URL: https://issues.apache.org/jira/browse/HBASE-5923
Project: HBase
Issue Type: Improvement
Components: Client, regionserver
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Critical
Fix For: 0.95.0
Attachments: 5923-0.94.txt, 5923-trunk.txt
1. the checkAnd{Put|Delete} method that takes a CompareOP is not exposed via
HTable[Interface].
2. there is unnecessary duplicate code in the check{Put|Delete} code in
HRegionServer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619261#comment-13619261 ] Andrew Purtell commented on HBASE-7904: --- I'm going to close this issue. Let's deal with the individual items here with JIRAs that focus on them individually. Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Priority: Critical Fix For: 0.95.0, 0.98.0 Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-7904. --- Resolution: Invalid Fix Version/s: (was: 0.95.0) (was: 0.98.0) Assignee: (was: Ted Yu) Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Reporter: Ted Yu Priority: Critical Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell closed HBASE-7904. - Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Reporter: Ted Yu Priority: Critical Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8224) Add '-hadoop1' or '-hadoop2' to our version string
[ https://issues.apache.org/jira/browse/HBASE-8224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619269#comment-13619269 ] Enis Soztutar commented on HBASE-8224: -- He was talking about the artifact name containing the hadoop1 or 2 suffix, instead of the version. It is close to the one I was talking about above, where we have two poms per module. The module artifacts will be named hbase-client-hadoop1, and the version will still be 0.95.0. So the final artifact name will be hbase-client-hadoop1-0.95.0.jar. We can supply an alternate file using mvn -f flag. But having the build is already modular, we have to test it. Add '-hadoop1' or '-hadoop2' to our version string -- Key: HBASE-8224 URL: https://issues.apache.org/jira/browse/HBASE-8224 Project: HBase Issue Type: Task Reporter: stack Attachments: 8224-adding.classifiers.txt So we can publish both the hadoop1 and the hadoop2 jars to a maven repository, and so we can publish two packages, one for hadoop1 and one for hadoop2, given how maven works, our only alternative (to the best of my knowledge and after consulting others) is by amending the version string to include hadoop1 or hadoop2. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7904: Component/s: mapreduce Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Components: mapreduce Reporter: Ted Yu Priority: Critical Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619284#comment-13619284 ] Ted Yu commented on HBASE-7904: --- YARN-449, marked Resolved, is linked to this issue. Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Components: mapreduce Reporter: Ted Yu Priority: Critical Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619286#comment-13619286 ] Andrew Purtell commented on HBASE-7904: --- Unlinked from YARN-449. Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Components: mapreduce Reporter: Ted Yu Priority: Critical Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5746) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums (0.96)
[ https://issues.apache.org/jira/browse/HBASE-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619288#comment-13619288 ] Sergey Shelukhin commented on HBASE-5746: - Let me try to rebase this onto trunk... there appear to be some impossible dependencies so things will need to move between modules HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums (0.96) - Key: HBASE-5746 URL: https://issues.apache.org/jira/browse/HBASE-5746 Project: HBase Issue Type: Sub-task Components: io, regionserver Reporter: Lars Hofhansl Priority: Critical Fix For: 0.95.0 Attachments: 5720-trunk-v2.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3967) Support deletes in HFileOutputFormat based bulk import mechanism
[ https://issues.apache.org/jira/browse/HBASE-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619292#comment-13619292 ] Sergey Shelukhin commented on HBASE-3967: - Should this be closed as invalid then? Support deletes in HFileOutputFormat based bulk import mechanism Key: HBASE-3967 URL: https://issues.apache.org/jira/browse/HBASE-3967 Project: HBase Issue Type: Sub-task Components: mapreduce Reporter: Kannan Muthukkaruppan Priority: Critical Fix For: 0.95.0 Attachments: 3967-janky-verify.patch, diff.patch During bulk imports, it'll be useful to be able to do delete mutations (either to delete data that already exists in HBase or was inserted earlier during this run of the import). For example, we have a use case, where we are processing a log of data which may have both inserts and deletes in the mix and we want to upload that into HBase using the bulk import mechanism. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3643) Close the filesystem handle when HRS is aborting
[
https://issues.apache.org/jira/browse/HBASE-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619302#comment-13619302
]
Sergey Shelukhin commented on HBASE-3643:
-
I don't quite understand this JIRA... what kind of damage is done? I am
assuming these are all the same operations that would have happened anyway on
other threads if we didn't call abort... in which case they should be
legitimate?
Close the filesystem handle when HRS is aborting
Key: HBASE-3643
URL: https://issues.apache.org/jira/browse/HBASE-3643
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.1
Reporter: Jean-Daniel Cryans
Priority: Critical
Fix For: 0.95.0
I thought of a way to fix HBASE-3515 that has a very broad impact, so I'm
creating this jira to *raise awareness* and gather comments.
Currently when we call HRS.abort, it's still possible to do HDFS operations
like rolling logs and flushing files. It also has the impact that some
threads cannot write to ZK (like the situation described in HBASE-3515) but
then can still write to HDFS. Since that call is so central, I think we
should {color:red} add fs.close() inside the abort method{color}.
The impact of this is that everything else that happens after the close call,
like closing files or appending, will fail in the most horrible ways. On the
bright side, this means less disruptive changes on HDFS.
Todd pointed at HBASE-2231 as related, but I think my solution is still too
sloppy as we could still finish a compaction and immediately close the
filesystem after that (damage's done).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7605) TestMiniClusterLoadSequential fails in trunk build on hadoop 2
[ https://issues.apache.org/jira/browse/HBASE-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619311#comment-13619311 ] Sergey Shelukhin commented on HBASE-7605: - is this still relevant? The last error seems env-related. TestMiniClusterLoadSequential fails in trunk build on hadoop 2 -- Key: HBASE-7605 URL: https://issues.apache.org/jira/browse/HBASE-7605 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Priority: Critical Fix For: 0.95.0 From HBase-TRUNK-on-Hadoop-2.0.0 #354: loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[1](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[2](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[3](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7605) TestMiniClusterLoadSequential fails in trunk build on hadoop 2
[ https://issues.apache.org/jira/browse/HBASE-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619312#comment-13619312 ] Ted Yu commented on HBASE-7605: --- The test failed in https://builds.apache.org/job/hbase-0.95-on-hadoop2/51/#showFailuresLink TestMiniClusterLoadSequential fails in trunk build on hadoop 2 -- Key: HBASE-7605 URL: https://issues.apache.org/jira/browse/HBASE-7605 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Priority: Critical Fix For: 0.95.0 From HBase-TRUNK-on-Hadoop-2.0.0 #354: loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[1](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[2](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[3](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8127) Region of a disabling or disabled table could be stuck in transition state when RS dies during Master initialization
[
https://issues.apache.org/jira/browse/HBASE-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619314#comment-13619314
]
Ted Yu commented on HBASE-8127:
---
Test suite is green for patch v2.
[~jxiang], [~ram_krish], [~lhofhansl]:
Do you want to take one more look ?
Region of a disabling or disabled table could be stuck in transition state
when RS dies during Master initialization
Key: HBASE-8127
URL: https://issues.apache.org/jira/browse/HBASE-8127
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.5
Reporter: Jeffrey Zhong
Assignee: rajeshbabu
Fix For: 0.94.7
Attachments: HBASE-8127_94_2.patch, HBASE-8127_94_3.patch,
HBASE-8127_feedback.patch, HBASE-8127.patch, hbase-8127_v1.patch,
reproduce-hang.patch
The issue happens when a RS dies during a master starts up. After the RS
reports open to the new master instance and dies immediately thereafter, the
RITs of disabling tables(or disabled table) on the died RS will be in RIT
state forever.
I attached a patch to simulate the situation and you can run the following
command to reproduce the issue:
{code}mvn test -PlocalTests
-Dtest=TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS{code}
Basically, we skip regions of a dead server inside
AM.processDeadServersAndRecoverLostRegions as the following code and relies
on SSH to process those skipped regions:
{code}
for (PairHRegionInfo, Result deadRegion : deadServer.getValue()) {
nodes.remove(deadRegion.getFirst().getEncodedName());
}
{code}
While in SSH, we skip regions of disabling(or disabled table) again by
function processDeadRegion. Finally comes to the issue that RITs of
disabling(or disabled table) stuck there forever.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7605) TestMiniClusterLoadSequential fails in trunk build on hadoop 2
[ https://issues.apache.org/jira/browse/HBASE-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619315#comment-13619315 ] Andrew Purtell commented on HBASE-7605: --- Passing here: http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/org.apache.hbase$hbase-server/52/testReport/org.apache.hadoop.hbase.util/TestMiniClusterLoadSequential/ TestMiniClusterLoadSequential fails in trunk build on hadoop 2 -- Key: HBASE-7605 URL: https://issues.apache.org/jira/browse/HBASE-7605 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Priority: Critical Fix For: 0.95.0 From HBase-TRUNK-on-Hadoop-2.0.0 #354: loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[1](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[2](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[3](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7605) TestMiniClusterLoadSequential fails in trunk build on hadoop 2
[ https://issues.apache.org/jira/browse/HBASE-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619317#comment-13619317 ] Ted Yu commented on HBASE-7605: --- But that was before HBASE-7904 being reverted. TestMiniClusterLoadSequential fails in trunk build on hadoop 2 -- Key: HBASE-7605 URL: https://issues.apache.org/jira/browse/HBASE-7605 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Priority: Critical Fix For: 0.95.0 From HBase-TRUNK-on-Hadoop-2.0.0 #354: loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[1](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[2](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[3](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3967) Support deletes in HFileOutputFormat based bulk import mechanism
[ https://issues.apache.org/jira/browse/HBASE-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619320#comment-13619320 ] Ted Yu commented on HBASE-3967: --- The diff.patch was for 0.89-fb branch. I don't think this should be closed as being invalid. Support deletes in HFileOutputFormat based bulk import mechanism Key: HBASE-3967 URL: https://issues.apache.org/jira/browse/HBASE-3967 Project: HBase Issue Type: Sub-task Components: mapreduce Reporter: Kannan Muthukkaruppan Priority: Critical Fix For: 0.95.0 Attachments: 3967-janky-verify.patch, diff.patch During bulk imports, it'll be useful to be able to do delete mutations (either to delete data that already exists in HBase or was inserted earlier during this run of the import). For example, we have a use case, where we are processing a log of data which may have both inserts and deletes in the mix and we want to upload that into HBase using the bulk import mechanism. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5746) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums (0.96)
[ https://issues.apache.org/jira/browse/HBASE-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HBASE-5746: --- Assignee: Sergey Shelukhin HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums (0.96) - Key: HBASE-5746 URL: https://issues.apache.org/jira/browse/HBASE-5746 Project: HBase Issue Type: Sub-task Components: io, regionserver Reporter: Lars Hofhansl Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.95.0 Attachments: 5720-trunk-v2.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8000) create integration/perf tests for stripe compactions
[ https://issues.apache.org/jira/browse/HBASE-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-8000: Attachment: HBASE-8000-v0-with-7679-7680-7967.patch HBASE-8000-v0.patch Proper patch create integration/perf tests for stripe compactions Key: HBASE-8000 URL: https://issues.apache.org/jira/browse/HBASE-8000 Project: HBase Issue Type: Sub-task Components: Compaction Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HBASE-8000-v0.patch, HBASE-8000-v0-with-7679-7680-7967.patch While writing tests I seem to be finding more errors with edge cases unrelated to what test actually tests compared to what is expected. Integration test will be needed... probably good for perf/compare too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8000) create integration/perf tests for stripe compactions
[ https://issues.apache.org/jira/browse/HBASE-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-8000: Attachment: (was: HBASE-8000-v-minus0.1.patch) create integration/perf tests for stripe compactions Key: HBASE-8000 URL: https://issues.apache.org/jira/browse/HBASE-8000 Project: HBase Issue Type: Sub-task Components: Compaction Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HBASE-8000-v0.patch, HBASE-8000-v0-with-7679-7680-7967.patch While writing tests I seem to be finding more errors with edge cases unrelated to what test actually tests compared to what is expected. Integration test will be needed... probably good for perf/compare too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8000) create integration/perf tests for stripe compactions
[ https://issues.apache.org/jira/browse/HBASE-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-8000: Attachment: (was: HBASE-8000-v-minus0.1-with-7679-7680-7967.patch) create integration/perf tests for stripe compactions Key: HBASE-8000 URL: https://issues.apache.org/jira/browse/HBASE-8000 Project: HBase Issue Type: Sub-task Components: Compaction Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HBASE-8000-v0.patch, HBASE-8000-v0-with-7679-7680-7967.patch While writing tests I seem to be finding more errors with edge cases unrelated to what test actually tests compared to what is expected. Integration test will be needed... probably good for perf/compare too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8000) create integration/perf tests for stripe compactions
[ https://issues.apache.org/jira/browse/HBASE-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-8000: Status: Patch Available (was: Open) create integration/perf tests for stripe compactions Key: HBASE-8000 URL: https://issues.apache.org/jira/browse/HBASE-8000 Project: HBase Issue Type: Sub-task Components: Compaction Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HBASE-8000-v0.patch, HBASE-8000-v0-with-7679-7680-7967.patch While writing tests I seem to be finding more errors with edge cases unrelated to what test actually tests compared to what is expected. Integration test will be needed... probably good for perf/compare too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3643) Close the filesystem handle when HRS is aborting
[
https://issues.apache.org/jira/browse/HBASE-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619323#comment-13619323
]
Jean-Daniel Cryans commented on HBASE-3643:
---
The damage done is the situation described in HBASE-2231 where a compaction
would end after a RS comes out of GC and then it deletes the old files.
Close the filesystem handle when HRS is aborting
Key: HBASE-3643
URL: https://issues.apache.org/jira/browse/HBASE-3643
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.1
Reporter: Jean-Daniel Cryans
Priority: Critical
Fix For: 0.95.0
I thought of a way to fix HBASE-3515 that has a very broad impact, so I'm
creating this jira to *raise awareness* and gather comments.
Currently when we call HRS.abort, it's still possible to do HDFS operations
like rolling logs and flushing files. It also has the impact that some
threads cannot write to ZK (like the situation described in HBASE-3515) but
then can still write to HDFS. Since that call is so central, I think we
should {color:red} add fs.close() inside the abort method{color}.
The impact of this is that everything else that happens after the close call,
like closing files or appending, will fail in the most horrible ways. On the
bright side, this means less disruptive changes on HDFS.
Todd pointed at HBASE-2231 as related, but I think my solution is still too
sloppy as we could still finish a compaction and immediately close the
filesystem after that (damage's done).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8000) create integration/perf tests for stripe compactions
[ https://issues.apache.org/jira/browse/HBASE-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619325#comment-13619325 ] Sergey Shelukhin commented on HBASE-8000: - https://reviews.apache.org/r/10230/ create integration/perf tests for stripe compactions Key: HBASE-8000 URL: https://issues.apache.org/jira/browse/HBASE-8000 Project: HBase Issue Type: Sub-task Components: Compaction Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HBASE-8000-v0.patch, HBASE-8000-v0-with-7679-7680-7967.patch While writing tests I seem to be finding more errors with edge cases unrelated to what test actually tests compared to what is expected. Integration test will be needed... probably good for perf/compare too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8224) Add '-hadoop1' or '-hadoop2' to our version string
[
https://issues.apache.org/jira/browse/HBASE-8224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-8224:
-
Attachment: hbase-8224-proto1.patch
[~stack] could you please take a quick look at the prototype patch.
{code}
mvn clean install -DskipTests
{code}
builds with hadoop1 suffix:
{code}
[INFO] Installing
/Users/enis/projects/svn-repos/hbase/hbase-server/target/hbase-server-hadoop1-0.97-SNAPSHOT.jar
to
/Users/enis/.m2/repository/org/apache/hbase/hbase-server-hadoop1/0.97-SNAPSHOT/hbase-server-hadoop1-0.97-SNAPSHOT.jar
[INFO] Installing /Users/enis/projects/svn-repos/hbase/hbase-server/pom.xml to
/Users/enis/.m2/repository/org/apache/hbase/hbase-server-hadoop1/0.97-SNAPSHOT/hbase-server-hadoop1-0.97-SNAPSHOT.pom
[INFO] Installing
/Users/enis/projects/svn-repos/hbase/hbase-server/target/hbase-server-hadoop1-0.97-SNAPSHOT-tests.jar
to
/Users/enis/.m2/repository/org/apache/hbase/hbase-server-hadoop1/0.97-SNAPSHOT/hbase-server-hadoop1-0.97-SNAPSHOT-tests.jar
[INFO] Installing
/Users/enis/projects/svn-repos/hbase/hbase-server/target/hbase-server-hadoop1-0.97-SNAPSHOT-sources.jar
to
/Users/enis/.m2/repository/org/apache/hbase/hbase-server-hadoop1/0.97-SNAPSHOT/hbase-server-hadoop1-0.97-SNAPSHOT-sources.jar
{code}
For hadoop2 profile:
{code}
mvn install -DskipTests -Dhadoop.profile=2.0
{code}
results in
{code}
[INFO] Installing
/Users/enis/projects/svn-repos/hbase/hbase-server/target/hbase-server-hadoop2-0.97-SNAPSHOT.jar
to
/Users/enis/.m2/repository/org/apache/hbase/hbase-server-hadoop2/0.97-SNAPSHOT/hbase-server-hadoop2-0.97-SNAPSHOT.jar
[INFO] Installing /Users/enis/projects/svn-repos/hbase/hbase-server/pom.xml to
/Users/enis/.m2/repository/org/apache/hbase/hbase-server-hadoop2/0.97-SNAPSHOT/hbase-server-hadoop2-0.97-SNAPSHOT.pom
[INFO] Installing
/Users/enis/projects/svn-repos/hbase/hbase-server/target/hbase-server-hadoop2-0.97-SNAPSHOT-tests.jar
to
/Users/enis/.m2/repository/org/apache/hbase/hbase-server-hadoop2/0.97-SNAPSHOT/hbase-server-hadoop2-0.97-SNAPSHOT-tests.jar
[INFO] Installing
/Users/enis/projects/svn-repos/hbase/hbase-server/target/hbase-server-hadoop2-0.97-SNAPSHOT-sources.jar
to
/Users/enis/.m2/repository/org/apache/hbase/hbase-server-hadoop2/0.97-SNAPSHOT/hbase-server-hadoop2-0.97-SNAPSHOT-sources.jar
{code}
Not sure whether this will work with the assembly or maven publish, but install
seems fine.
Add '-hadoop1' or '-hadoop2' to our version string
--
Key: HBASE-8224
URL: https://issues.apache.org/jira/browse/HBASE-8224
Project: HBase
Issue Type: Task
Reporter: stack
Attachments: 8224-adding.classifiers.txt, hbase-8224-proto1.patch
So we can publish both the hadoop1 and the hadoop2 jars to a maven
repository, and so we can publish two packages, one for hadoop1 and one for
hadoop2, given how maven works, our only alternative (to the best of my
knowledge and after consulting others) is by amending the version string to
include hadoop1 or hadoop2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8224) Add '-hadoop1' or '-hadoop2' to our version string
[ https://issues.apache.org/jira/browse/HBASE-8224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619349#comment-13619349 ] stack commented on HBASE-8224: -- Oh, interpolating into artifactid works? Let me mess with it... will report back. Thanks [~enis] Add '-hadoop1' or '-hadoop2' to our version string -- Key: HBASE-8224 URL: https://issues.apache.org/jira/browse/HBASE-8224 Project: HBase Issue Type: Task Reporter: stack Attachments: 8224-adding.classifiers.txt, hbase-8224-proto1.patch So we can publish both the hadoop1 and the hadoop2 jars to a maven repository, and so we can publish two packages, one for hadoop1 and one for hadoop2, given how maven works, our only alternative (to the best of my knowledge and after consulting others) is by amending the version string to include hadoop1 or hadoop2. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7836) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics
[
https://issues.apache.org/jira/browse/HBASE-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeffrey Zhong updated HBASE-7836:
-
Attachment: hbase-7836_v2.patch
{quote}
testWorkerAbort(org.apache.hadoop.hbase.master.TestDistributedLogSplitting):
none of the following counters went up in 8 milliseconds -
tot_wkr_task_resigned, tot_wkr_task_err, tot_wkr_final_transition_failed,
tot_wkr_task_done, tot_wkr_preempt_task
{quote}
This is due to we don't handle FSHDFSUtils.recoverFileLease
java.nio.channels.ClosedByInterruptException. When we get this exception, we
still call {code}FSDataOutputStream out = fs.append(p);{code}, that causes one
extra min wait and then fails the test case due to timeout. Below are related
log traces:
{code}
2013-04-01 14:45:04,735 DEBUG [SplitLogWorker-10.11.2.103,58161,1364852631051]
util.FSHDFSUtils(95): Failed fs.recoverLease invocation, java.io.IOException:
Call to localhost/127.0.0.1:58147 failed on local exception:
java.nio.channels.ClosedByInterruptException, trying fs.append instead
2013-04-01 14:45:04,735 DEBUG [SplitLogWorker-10.11.2.103,58161,1364852631051]
util.FSHDFSUtils(100): trying fs.append for
hdfs://localhost:58147/user/jzhong/hbase/.logs/10.11.2.103,58161,1364852631051/10.11.2.103%2C58161%2C1364852631051.1364852632043
with java.io.IOException: Call to localhost/127.0.0.1:58147 failed on local
exception: java.nio.channels.ClosedByInterruptException
...
2013-04-01 14:46:04,737 WARN [SplitLogWorker-10.11.2.103,58161,1364852631051]
regionserver.SplitLogWorker$1(124): log splitting of
hdfs://localhost:58147/user/jzhong/hbase/.logs/10.11.2.103,58161,1364852631051/10.11.2.103%2C58161%2C1364852631051.1364852632043
failed, returning error
java.io.IOException: Failed to open
hdfs://localhost:58147/user/jzhong/hbase/.logs/10.11.2.103,58161,1364852631051/10.11.2.103%2C58161%2C1364852631051.1364852632043
for append
at
org.apache.hadoop.hbase.util.FSHDFSUtils.recoverFileLease(FSHDFSUtils.java:126)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getReader(HLogSplitter.java:743)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:436)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:397)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:111)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.grabTask(SplitLogWorker.java:274)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.taskLoop(SplitLogWorker.java:195)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.run(SplitLogWorker.java:162)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.io.IOException: Call to localhost/127.0.0.1:58147 failed on
local exception: java.nio.channels.ClosedByInterruptException
at org.apache.hadoop.ipc.Client.wrapException(Client.java:1144)
at org.apache.hadoop.ipc.Client.call(Client.java:1112)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
….
{code}
I fixed other test failures in the new patch.
Thanks,
-Jeffrey
Create a new replay command so that recovered edits won't mess up normal
coprocessing metrics
-
Key: HBASE-7836
URL: https://issues.apache.org/jira/browse/HBASE-7836
Project: HBase
Issue Type: Sub-task
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.95.0
Attachments: hbase-7836_v1.patch, hbase-7836_v2.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8224) Add '-hadoop1' or '-hadoop2' to our version string
[
https://issues.apache.org/jira/browse/HBASE-8224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619352#comment-13619352
]
stack commented on HBASE-8224:
--
Here is what I did making the release just now:
{code}
2 3491 rm -rf ~/Downloads/hbase-0.95.0/*
3 3490 mvn clean -Prelease -Dassembly.file=src/assembly/src.xml install
-DskipTests assembly:single
4 3492 cp hbase-assembly/target/hbase-0.95.0-src.tar.gz
~/Downloads/hbase-0.95.0/
5 3493 svn revert -R .
6 3494 svn cleanup
7 3495 mvn clean org.codehaus.mojo:versions-maven-plugin:1.3.1:set
-DnewVersion=0.95.0-hadoop1
8 3496 mvn clean -Prelease install -DskipTests javadoc:aggregate site
assembly:single
9 3497 cp hbase-assembly/target/hbase-0.95.0-hadoop1-bin.tar.gz
~/Downloads/hbase-0.95.0/
10 3498 svn revert -R .
11 3499 svn cleanup
12 3501 mvn clean -Prelease -Dhadoop.profile=2.0 install -DskipTests
javadoc:aggregate site assembly:single
13 3502 mv hbase-assembly/target/hbase-0.95.0-hadoop2-bin.tar.gz
~/Downloads/hbase-0.95.0/
14 3513 tail -f logs/hbase-stack-master-durruti.local.log.
15 3518 svn copy https://svn.apache.org/repos/asf/hbase/branches/0.95
https://svn.apache.org/repos/asf/hbase/tags/0.95.0RC0 -m Tag 0.95.0RC0
16 3520 cd Downloads/hbase-0.95.0/
17 3522 gpg --print-mds hbase-0.95.0-src.tar.gz hbase-0.95.0-src.tar.gz.mds
18 3524 gpg --print-mds hbase-0.95.0-hadoop1-bin.tar.gz
hbase-0.95.0-hadoop1-bin.tar.gz.mds
19 3525 gpg --print-mds hbase-0.95.0-hadoop2-bin.tar.gz
hbase-0.95.0-hadoop2-bin.tar.gz.mds
20 3527 gpg --armor --output hbase-0.95.0-src.tar.gz.asc --detach-sig
hbase-0.95.0-src.tar.gz
21 3530 gpg --armor --output hbase-0.95.0-hadoop1-bin.tar.gz.asc
--detach-sig hbase-0.95.0-hadoop1-bin.tar.gz
22 3531 gpg --armor --output hbase-0.95.0-hadoop2-bin.tar.gz.asc
--detach-sig hbase-0.95.0-hadoop2-bin.tar.gz
23 3533 cd ../
24 3534 mv hbase-0.95.0 hbase-0.95.0RC0
25 3535 tar cfz hbase-0.95.0RC0.tgz hbase-0.95.0RC0
26 3539 scp hbase-0.95.0RC0.tgz people.apache.org:
{code}
I set the version using mvn versions and then I'd do my build of src tgz, copy
it over to a 0.95 dir; then clean up the changes made by the mvn versions by
doing svn revert ; then run it again to set hadoop1, and do similar for
hadoop2.
Let me push up to mvn repotoo.
Add '-hadoop1' or '-hadoop2' to our version string
--
Key: HBASE-8224
URL: https://issues.apache.org/jira/browse/HBASE-8224
Project: HBase
Issue Type: Task
Reporter: stack
Attachments: 8224-adding.classifiers.txt, hbase-8224-proto1.patch
So we can publish both the hadoop1 and the hadoop2 jars to a maven
repository, and so we can publish two packages, one for hadoop1 and one for
hadoop2, given how maven works, our only alternative (to the best of my
knowledge and after consulting others) is by amending the version string to
include hadoop1 or hadoop2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[
https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619354#comment-13619354
]
Ted Yu commented on HBASE-7904:
---
I plan to introduce the following methods to CoprocessorEnvironment and :
{code}
/** @return the mutable configuration */
public Configuration getMutableConfiguration();
/** @return the immutable configuration */
public Configuration getImmutableConfiguration();
{code}
w.r.t. the existing method:
{code}
/** @return the configuration */
public Configuration getConfiguration();
{code}
We can either remove it or let it return the immutable config (current
behavior).
Make mapreduce jobs pass based on 2.0.4-alpha
-
Key: HBASE-7904
URL: https://issues.apache.org/jira/browse/HBASE-7904
Project: HBase
Issue Type: Task
Components: mapreduce
Reporter: Ted Yu
Priority: Critical
Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt,
7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt,
7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt,
7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt,
7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt
2.0.3-alpha has been released.
We should upgrade the dependency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[
https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619356#comment-13619356
]
Andrew Purtell commented on HBASE-7904:
---
{quote}
I plan to introduce the following methods to CoprocessorEnvironment and :
{noformat}
/** @return the mutable configuration */
public Configuration getMutableConfiguration();
/** @return the immutable configuration */
public Configuration getImmutableConfiguration();
{noformat}
{quote}
-1, see my above comment. I know you read it, why are you ignoring it?
-1, this issue is closed.
Make mapreduce jobs pass based on 2.0.4-alpha
-
Key: HBASE-7904
URL: https://issues.apache.org/jira/browse/HBASE-7904
Project: HBase
Issue Type: Task
Components: mapreduce
Reporter: Ted Yu
Priority: Critical
Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt,
7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt,
7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt,
7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt,
7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt
2.0.3-alpha has been released.
We should upgrade the dependency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8204) Logging improvements and try to recover lease even when append is not activated.
[
https://issues.apache.org/jira/browse/HBASE-8204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619360#comment-13619360
]
Jeffrey Zhong commented on HBASE-8204:
--
[~nkeywal] One motivation for you to remove the {code} FSDataOutputStream out =
fs.append(p);{code} from function recoverFileLease. Today I found the test case
TestDistributedLogSplitting#testWorkerAbort is flaky due to we don't handle
java.nio.channels.ClosedByInterruptException when recoverLease get
interrupted. Even after it's interrupted but it still calls fs.append which
result in an extra minute wait so fails the test case with timeout error. Below
are related traces:
{code}
2013-04-01 14:45:04,735 DEBUG [SplitLogWorker-10.11.2.103,58161,1364852631051]
util.FSHDFSUtils(95): Failed fs.recoverLease invocation, java.io.IOException:
Call to localhost/127.0.0.1:58147 failed on local exception:
java.nio.channels.ClosedByInterruptException, trying fs.append instead
2013-04-01 14:45:04,735 DEBUG [SplitLogWorker-10.11.2.103,58161,1364852631051]
util.FSHDFSUtils(100): trying fs.append for
hdfs://localhost:58147/user/jzhong/hbase/.logs/10.11.2.103,58161,1364852631051/10.11.2.103%2C58161%2C1364852631051.1364852632043
with java.io.IOException: Call to localhost/127.0.0.1:58147 failed on local
exception: java.nio.channels.ClosedByInterruptException
...
2013-04-01 14:46:04,737 WARN [SplitLogWorker-10.11.2.103,58161,1364852631051]
regionserver.SplitLogWorker$1(124): log splitting of
hdfs://localhost:58147/user/jzhong/hbase/.logs/10.11.2.103,58161,1364852631051/10.11.2.103%2C58161%2C1364852631051.1364852632043
failed, returning error
java.io.IOException: Failed to open
hdfs://localhost:58147/user/jzhong/hbase/.logs/10.11.2.103,58161,1364852631051/10.11.2.103%2C58161%2C1364852631051.1364852632043
for append
at
org.apache.hadoop.hbase.util.FSHDFSUtils.recoverFileLease(FSHDFSUtils.java:126)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getReader(HLogSplitter.java:743)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:436)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:397)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:111)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.grabTask(SplitLogWorker.java:274)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.taskLoop(SplitLogWorker.java:195)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.run(SplitLogWorker.java:162)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.io.IOException: Call to localhost/127.0.0.1:58147 failed on
local exception: java.nio.channels.ClosedByInterruptException
at org.apache.hadoop.ipc.Client.wrapException(Client.java:1144)
at org.apache.hadoop.ipc.Client.call(Client.java:1112)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
….
{code}
Logging improvements and try to recover lease even when append is not
activated.
Key: HBASE-8204
URL: https://issues.apache.org/jira/browse/HBASE-8204
Project: HBase
Issue Type: Bug
Components: wal
Affects Versions: 0.96.0
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Priority: Minor
Attachments: 8204.v2.patch, 8204.v2.patch, 8204.v3.patch,
HBASE-8204.v1.patch
See patch :-).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[
https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619363#comment-13619363
]
Ted Yu commented on HBASE-7904:
---
I think you were referring to
https://issues.apache.org/jira/browse/HBASE-7904?focusedCommentId=13619252page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13619252
SecureBulkLoadEndpoint needs a way to get access to both Configuration's, right
?
See the following in SecureBulkLoadEndpoint#start():
{code}
conf = env.getConfiguration();
{code}
Make mapreduce jobs pass based on 2.0.4-alpha
-
Key: HBASE-7904
URL: https://issues.apache.org/jira/browse/HBASE-7904
Project: HBase
Issue Type: Task
Components: mapreduce
Reporter: Ted Yu
Priority: Critical
Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt,
7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt,
7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt,
7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt,
7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt
2.0.3-alpha has been released.
We should upgrade the dependency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619366#comment-13619366 ] Andrew Purtell commented on HBASE-7904: --- -1 I won't discuss this further here, this issue is closed. Kindly open a JIRA to address the SecureBulkLoadEndpoint issue. Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Components: mapreduce Reporter: Ted Yu Priority: Critical Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619367#comment-13619367 ] Ted Yu commented on HBASE-7904: --- @Andy: Maybe you're clear about the new JIRA (title, description, etc) If you can open the JIRA, that would be great. Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Components: mapreduce Reporter: Ted Yu Priority: Critical Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7605) TestMiniClusterLoadSequential fails in trunk build on hadoop 2
[ https://issues.apache.org/jira/browse/HBASE-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619368#comment-13619368 ] Andrew Purtell commented on HBASE-7605: --- bq. But that was before HBASE-7904 being reverted. Of course, thanks Ted. Yes it's failing and not due to an env issue. http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/org.apache.hbase$hbase-server/53/testReport/ TestMiniClusterLoadSequential fails in trunk build on hadoop 2 -- Key: HBASE-7605 URL: https://issues.apache.org/jira/browse/HBASE-7605 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Priority: Critical Fix For: 0.95.0 From HBase-TRUNK-on-Hadoop-2.0.0 #354: loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[1](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[2](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[3](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619370#comment-13619370 ] Andrew Purtell commented on HBASE-7904: --- bq. If you can open the JIRA, that would be great. Surely this is an April Fool's joke and not a display of petulance. Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Components: mapreduce Reporter: Ted Yu Priority: Critical Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8000) create integration/perf tests for stripe compactions
[
https://issues.apache.org/jira/browse/HBASE-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619378#comment-13619378
]
Hadoop QA commented on HBASE-8000:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12576455/HBASE-8000-v0-with-7679-7680-7967.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 39 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 7
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestHLog
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5086//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5086//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5086//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5086//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5086//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5086//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5086//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5086//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5086//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5086//console
This message is automatically generated.
create integration/perf tests for stripe compactions
Key: HBASE-8000
URL: https://issues.apache.org/jira/browse/HBASE-8000
Project: HBase
Issue Type: Sub-task
Components: Compaction
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: HBASE-8000-v0.patch,
HBASE-8000-v0-with-7679-7680-7967.patch
While writing tests I seem to be finding more errors with edge cases
unrelated to what test actually tests compared to what is expected.
Integration test will be needed... probably good for perf/compare too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7667) Support stripe compaction
[ https://issues.apache.org/jira/browse/HBASE-7667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619380#comment-13619380 ] Sergey Shelukhin commented on HBASE-7667: - Actually, judging by logs what can be done is triggering compaction thread if store can compact. In 25-stripe case I see gaps between compactions which are unnecessary, when the compaction only triggers on flush despite plenty of tiny stripe compactions being possible Support stripe compaction - Key: HBASE-7667 URL: https://issues.apache.org/jira/browse/HBASE-7667 Project: HBase Issue Type: New Feature Components: Compaction Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: Stripe compaction perf evaluation.pdf, Stripe compaction perf evaluation.pdf, Stripe compactions.pdf, Stripe compactions.pdf, Stripe compactions.pdf So I was thinking about having many regions as the way to make compactions more manageable, and writing the level db doc about how level db range overlap and data mixing breaks seqNum sorting, and discussing it with Jimmy, Matteo and Ted, and thinking about how to avoid Level DB I/O multiplication factor. And I suggest the following idea, let's call it stripe compactions. It's a mix between level db ideas and having many small regions. It allows us to have a subset of benefits of many regions (wrt reads and compactions) without many of the drawbacks (managing and current memstore/etc. limitation). It also doesn't break seqNum-based file sorting for any one key. It works like this. The region key space is separated into configurable number of fixed-boundary stripes (determined the first time we stripe the data, see below). All the data from memstores is written to normal files with all keys present (not striped), similar to L0 in LevelDb, or current files. Compaction policy does 3 types of compactions. First is L0 compaction, which takes all L0 files and breaks them down by stripe. It may be optimized by adding more small files from different stripes, but the main logical outcome is that there are no more L0 files and all data is striped. Second is exactly similar to current compaction, but compacting one single stripe. In future, nothing prevents us from applying compaction rules and compacting part of the stripe (e.g. similar to current policy with rations and stuff, tiers, whatever), but for the first cut I'd argue let it major compact the entire stripe. Or just have the ratio and no more complexity. Finally, the third addresses the concern of the fixed boundaries causing stripes to be very unbalanced. It's exactly like the 2nd, except it takes 2+ adjacent stripes and writes the results out with different boundaries. There's a tradeoff here - if we always take 2 adjacent stripes, compactions will be smaller but rebalancing will take ridiculous amount of I/O. If we take many stripes we are essentially getting into the epic-major-compaction problem again. Some heuristics will have to be in place. In general, if, before stripes are determined, we initially let L0 grow before determining the stripes, we will get better boundaries. Also, unless unbalancing is really large we don't need to rebalance really. Obviously this scheme (as well as level) is not applicable for all scenarios, e.g. if timestamp is your key it completely falls apart. The end result: - many small compactions that can be spread out in time. - reads still read from a small number of files (one stripe + L0). - region splits become marvelously simple (if we could move files between regions, no references would be needed). Main advantage over Level (for HBase) is that default store can still open the files and get correct results - there are no range overlap shenanigans. It also needs no metadata, although we may record some for convenience. It also would appear to not cause as much I/O. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8224) Add '-hadoop1' or '-hadoop2' to our version string
[
https://issues.apache.org/jira/browse/HBASE-8224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619384#comment-13619384
]
Enis Soztutar commented on HBASE-8224:
--
I did a bit more digging, but ran into a fundamental problem. Overriding the
artifactId from a property coming from the profile works, but when a project
depends on the installed artifact:
{code}
dependency
groupIdorg.apache.hbase/groupId
artifactIdhbase-server-hadoop2/artifactId
version0.97-SNAPSHOT/version
/dependency
/dependencies
{code}
then, there is no profile information, so it will just try to fetch the jars
from hadoop1 versions. I tried to circumvent this with putting a parent pom for
hadoop2 profile (pom-hadoop2.xml) which is a child of pom.xml, but also
declares child modules hbase-client, etc. This also fails to work because the
parent artifactId does not accept variables.
Add '-hadoop1' or '-hadoop2' to our version string
--
Key: HBASE-8224
URL: https://issues.apache.org/jira/browse/HBASE-8224
Project: HBase
Issue Type: Task
Reporter: stack
Attachments: 8224-adding.classifiers.txt, hbase-8224-proto1.patch
So we can publish both the hadoop1 and the hadoop2 jars to a maven
repository, and so we can publish two packages, one for hadoop1 and one for
hadoop2, given how maven works, our only alternative (to the best of my
knowledge and after consulting others) is by amending the version string to
include hadoop1 or hadoop2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8213) global authorization may lose efficacy
[
https://issues.apache.org/jira/browse/HBASE-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619386#comment-13619386
]
Jieshan Bean commented on HBASE-8213:
-
Thank you for the review, [~apurtell][~yuzhih...@gmail.com]:)
global authorization may lose efficacy
---
Key: HBASE-8213
URL: https://issues.apache.org/jira/browse/HBASE-8213
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.95.0, 0.96.0, 0.94.6
Reporter: Jieshan Bean
Assignee: Jieshan Bean
Priority: Critical
Fix For: 0.95.0, 0.98.0, 0.94.7
Attachments: HBASE-8213-94.patch, HBASE-8213-trunk.patch
It depends on the order of which region be opened first.
Suppose we have one 1 regionserver and only 1 user region REGION-A on this
server, _acl_ region was on another regionserver. _acl_ was opened a few
seconds before REGION-A.
The global authorization data read from Zookeeper was overwritten by the data
read from configuration.
{code}
private TableAuthManager(ZooKeeperWatcher watcher, Configuration conf)
throws IOException {
this.conf = conf;
this.zkperms = new ZKPermissionWatcher(watcher, this, conf);
try {
// Read global authorization data from zookeeper.
this.zkperms.start();
} catch (KeeperException ke) {
LOG.error(ZooKeeper initialization failed, ke);
}
// It will overwrite globalCache.
// initialize global permissions based on configuration
globalCache = initGlobal(conf);
}
{code}
This issue can be easily reproduced by below steps:
1. Start a cluster with 3 regionservers.
2. Create a new table T1.
3. grant a new user USER-A with global authorization.
4. Kill 1 regionserver RS3 and switch balance off.
5. Start regionserver RS3.
6. Assign region T1 to RS3.
7. Put data with user USER-A.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5746) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums (0.96)
[ https://issues.apache.org/jira/browse/HBASE-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-5746: Attachment: HBASE-5746-v0.patch rebased patch for trunk, fixed tests HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums (0.96) - Key: HBASE-5746 URL: https://issues.apache.org/jira/browse/HBASE-5746 Project: HBase Issue Type: Sub-task Components: io, regionserver Reporter: Lars Hofhansl Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.95.0 Attachments: 5720-trunk-v2.txt, HBASE-5746-v0.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5746) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums (0.96)
[ https://issues.apache.org/jira/browse/HBASE-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-5746: Status: Patch Available (was: Open) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums (0.96) - Key: HBASE-5746 URL: https://issues.apache.org/jira/browse/HBASE-5746 Project: HBase Issue Type: Sub-task Components: io, regionserver Reporter: Lars Hofhansl Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.95.0 Attachments: 5720-trunk-v2.txt, HBASE-5746-v0.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8239) ChaosMonkey actions for root and meta don't work
Sergey Shelukhin created HBASE-8239: --- Summary: ChaosMonkey actions for root and meta don't work Key: HBASE-8239 URL: https://issues.apache.org/jira/browse/HBASE-8239 Project: HBase Issue Type: Bug Components: test Reporter: Sergey Shelukhin Priority: Minor 1) Meta action doesn't work, it always fails to find the server holding meta, which has been the case for some time now. 2) Root action is no longer relevant: 13/04/01 20:36:25 INFO util.ChaosMonkey: Performing action: Restart region server holding ROOT org.apache.hadoop.hbase.exceptions.TableNotFoundException: Cannot find row in .META. for table: -ROOT-, row=-ROOT-,,99 at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:164) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8230) Possible NPE on regionserver abort if replication service has not been started
[
https://issues.apache.org/jira/browse/HBASE-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jieshan Bean updated HBASE-8230:
Attachment: HBASE-8230-trunk.patch
Patch for trunk.
Possible NPE on regionserver abort if replication service has not been started
--
Key: HBASE-8230
URL: https://issues.apache.org/jira/browse/HBASE-8230
Project: HBase
Issue Type: Bug
Components: regionserver, Replication
Affects Versions: 0.94.6
Reporter: Jieshan Bean
Assignee: Jieshan Bean
Attachments: HBASE-8230-94.patch, HBASE-8230-trunk.patch
RegionServer got Exception on calling setupWALAndReplication, so entered
abort flow. Since replicationSink had not been inialized yet, we got below
exception:
{noformat}
Exception in thread regionserver26003 java.lang.NullPointerException
at
org.apache.hadoop.hbase.replication.regionserver.Replication.join(Replication.java:129)
at
org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:120)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.join(HRegionServer.java:1803)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:834)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8230) Possible NPE on regionserver abort if replication service has not been started
[
https://issues.apache.org/jira/browse/HBASE-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jieshan Bean updated HBASE-8230:
Status: Patch Available (was: Open)
Possible NPE on regionserver abort if replication service has not been started
--
Key: HBASE-8230
URL: https://issues.apache.org/jira/browse/HBASE-8230
Project: HBase
Issue Type: Bug
Components: regionserver, Replication
Affects Versions: 0.94.6
Reporter: Jieshan Bean
Assignee: Jieshan Bean
Attachments: HBASE-8230-94.patch, HBASE-8230-trunk.patch
RegionServer got Exception on calling setupWALAndReplication, so entered
abort flow. Since replicationSink had not been inialized yet, we got below
exception:
{noformat}
Exception in thread regionserver26003 java.lang.NullPointerException
at
org.apache.hadoop.hbase.replication.regionserver.Replication.join(Replication.java:129)
at
org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:120)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.join(HRegionServer.java:1803)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:834)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8127) Region of a disabling or disabled table could be stuck in transition state when RS dies during Master initialization
[
https://issues.apache.org/jira/browse/HBASE-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619400#comment-13619400
]
rajeshbabu commented on HBASE-8127:
---
@Ted,
bq.Do we need to pass hri to the above method ? We can obtain this information
through rit, right ?
During master startup rit will be null then SSH will try to assign disabling
table regions present on the dead server. hri will help to avoid it.
Region of a disabling or disabled table could be stuck in transition state
when RS dies during Master initialization
Key: HBASE-8127
URL: https://issues.apache.org/jira/browse/HBASE-8127
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.5
Reporter: Jeffrey Zhong
Assignee: rajeshbabu
Fix For: 0.94.7
Attachments: HBASE-8127_94_2.patch, HBASE-8127_94_3.patch,
HBASE-8127_feedback.patch, HBASE-8127.patch, hbase-8127_v1.patch,
reproduce-hang.patch
The issue happens when a RS dies during a master starts up. After the RS
reports open to the new master instance and dies immediately thereafter, the
RITs of disabling tables(or disabled table) on the died RS will be in RIT
state forever.
I attached a patch to simulate the situation and you can run the following
command to reproduce the issue:
{code}mvn test -PlocalTests
-Dtest=TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS{code}
Basically, we skip regions of a dead server inside
AM.processDeadServersAndRecoverLostRegions as the following code and relies
on SSH to process those skipped regions:
{code}
for (PairHRegionInfo, Result deadRegion : deadServer.getValue()) {
nodes.remove(deadRegion.getFirst().getEncodedName());
}
{code}
While in SSH, we skip regions of disabling(or disabled table) again by
function processDeadRegion. Finally comes to the issue that RITs of
disabling(or disabled table) stuck there forever.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7745) Import uses System.out.println instead of logging
[ https://issues.apache.org/jira/browse/HBASE-7745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gustavo Anatoly updated HBASE-7745: --- Attachment: HBASE-7745.patch Import uses System.out.println instead of logging - Key: HBASE-7745 URL: https://issues.apache.org/jira/browse/HBASE-7745 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Nick Dimiduk Priority: Trivial Attachments: HBASE-7745.patch Per the title. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8219) Align Offline Merge with Online Merge
[ https://issues.apache.org/jira/browse/HBASE-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619412#comment-13619412 ] chunhui shen commented on HBASE-8219: - bq.the other two regions are removed by using the archiver,are they empty after the compaction? what does the 'empty' mean? After the compaction, the two regions' storefiles are still in their directory. Then archiving these files (may be used for snapshot) by archiver Align Offline Merge with Online Merge - Key: HBASE-8219 URL: https://issues.apache.org/jira/browse/HBASE-8219 Project: HBase Issue Type: Task Components: regionserver Affects Versions: 0.95.0 Reporter: Matteo Bertozzi Assignee: chunhui shen Attachments: hbase-8219v1.patch, hbase-8219v2.patch After HBASE-7403 we now have two different tools for online and offline merge, and the result produced by the two are different. (the online one works with snapshots, the offline not) We should remove the offline one, or align it to the online code. Most of the offline code in HRegion.merge() can be replaced with the one in RegionMergeTransaction, used by the online version. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8235) Adding inmemory CF attribute to LoadTest and PerformanceEvaluation tool
[ https://issues.apache.org/jira/browse/HBASE-8235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619415#comment-13619415 ] Jean-Marc Spaggiari commented on HBASE-8235: If this something which can be done also in 0.94 and 0.95 to be able to compare the results between the different branches? Adding inmemory CF attribute to LoadTest and PerformanceEvaluation tool --- Key: HBASE-8235 URL: https://issues.apache.org/jira/browse/HBASE-8235 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.98.0 Adding the inmemory CF attribute to the LoadTestTool and PerfEvaluation tool will make it extensible to test such inmemory scenarios also. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8219) Align Offline Merge with Online Merge
[ https://issues.apache.org/jira/browse/HBASE-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-8219: Attachment: hbase-8219v3.patch Patch v3: take a check whether merged region has reference after the compaction Align Offline Merge with Online Merge - Key: HBASE-8219 URL: https://issues.apache.org/jira/browse/HBASE-8219 Project: HBase Issue Type: Task Components: regionserver Affects Versions: 0.95.0 Reporter: Matteo Bertozzi Assignee: chunhui shen Attachments: hbase-8219v1.patch, hbase-8219v2.patch, hbase-8219v3.patch After HBASE-7403 we now have two different tools for online and offline merge, and the result produced by the two are different. (the online one works with snapshots, the offline not) We should remove the offline one, or align it to the online code. Most of the offline code in HRegion.merge() can be replaced with the one in RegionMergeTransaction, used by the online version. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8230) Possible NPE on regionserver abort if replication service has not been started
[
https://issues.apache.org/jira/browse/HBASE-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619424#comment-13619424
]
Chris Trezzo commented on HBASE-8230:
-
+1 for both patches. Looks good!
Possible NPE on regionserver abort if replication service has not been started
--
Key: HBASE-8230
URL: https://issues.apache.org/jira/browse/HBASE-8230
Project: HBase
Issue Type: Bug
Components: regionserver, Replication
Affects Versions: 0.94.6
Reporter: Jieshan Bean
Assignee: Jieshan Bean
Attachments: HBASE-8230-94.patch, HBASE-8230-trunk.patch
RegionServer got Exception on calling setupWALAndReplication, so entered
abort flow. Since replicationSink had not been inialized yet, we got below
exception:
{noformat}
Exception in thread regionserver26003 java.lang.NullPointerException
at
org.apache.hadoop.hbase.replication.regionserver.Replication.join(Replication.java:129)
at
org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:120)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.join(HRegionServer.java:1803)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:834)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8127) Region of a disabling or disabled table could be stuck in transition state when RS dies during Master initialization
[
https://issues.apache.org/jira/browse/HBASE-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619426#comment-13619426
]
Lars Hofhansl commented on HBASE-8127:
--
I don't know this part of the code well. Looks fine on cursory inspection.
Region of a disabling or disabled table could be stuck in transition state
when RS dies during Master initialization
Key: HBASE-8127
URL: https://issues.apache.org/jira/browse/HBASE-8127
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.5
Reporter: Jeffrey Zhong
Assignee: rajeshbabu
Fix For: 0.94.7
Attachments: HBASE-8127_94_2.patch, HBASE-8127_94_3.patch,
HBASE-8127_feedback.patch, HBASE-8127.patch, hbase-8127_v1.patch,
reproduce-hang.patch
The issue happens when a RS dies during a master starts up. After the RS
reports open to the new master instance and dies immediately thereafter, the
RITs of disabling tables(or disabled table) on the died RS will be in RIT
state forever.
I attached a patch to simulate the situation and you can run the following
command to reproduce the issue:
{code}mvn test -PlocalTests
-Dtest=TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS{code}
Basically, we skip regions of a dead server inside
AM.processDeadServersAndRecoverLostRegions as the following code and relies
on SSH to process those skipped regions:
{code}
for (PairHRegionInfo, Result deadRegion : deadServer.getValue()) {
nodes.remove(deadRegion.getFirst().getEncodedName());
}
{code}
While in SSH, we skip regions of disabling(or disabled table) again by
function processDeadRegion. Finally comes to the issue that RITs of
disabling(or disabled table) stuck there forever.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5746) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums (0.96)
[
https://issues.apache.org/jira/browse/HBASE-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619428#comment-13619428
]
Hadoop QA commented on HBASE-5746:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12576470/HBASE-5746-v0.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 12 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5087//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5087//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5087//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5087//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5087//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5087//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5087//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5087//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5087//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5087//console
This message is automatically generated.
HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no
checksums (0.96)
-
Key: HBASE-5746
URL: https://issues.apache.org/jira/browse/HBASE-5746
Project: HBase
Issue Type: Sub-task
Components: io, regionserver
Reporter: Lars Hofhansl
Assignee: Sergey Shelukhin
Priority: Critical
Fix For: 0.95.0
Attachments: 5720-trunk-v2.txt, HBASE-5746-v0.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8225) [replication] minor code bug when registering ReplicationLogCleaner
[
https://issues.apache.org/jira/browse/HBASE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619437#comment-13619437
]
Hudson commented on HBASE-8225:
---
Integrated in HBase-TRUNK #4005 (See
[https://builds.apache.org/job/HBase-TRUNK/4005/])
HBASE-8225 [replication] minor code bug when registering
ReplicationLogCleaner (Jerry He) (Revision 1463283)
Result = FAILURE
sershe :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
[replication] minor code bug when registering ReplicationLogCleaner
---
Key: HBASE-8225
URL: https://issues.apache.org/jira/browse/HBASE-8225
Project: HBase
Issue Type: Bug
Components: Replication
Affects Versions: 0.94.6
Reporter: Jerry He
Assignee: Jerry He
Priority: Minor
Fix For: 0.95.0, 0.96.0, 0.94.7
Attachments: HBASE-8255-0.94.patch, HBASE-8255-trunk.patch
We try to register ReplicationLogCleaner by default. This is done by calling
Replication.decorateMasterConfiguration()in the master.
In the current Replication.decorateMasterConfiguration():
...
String plugins = conf.get(HBASE_MASTER_LOGCLEANER_PLUGINS);
if (!plugins.contains(ReplicationLogCleaner.class.toString())) {
conf.set(HBASE_MASTER_LOGCLEANER_PLUGINS,
plugins + , + ReplicationLogCleaner.class.getCanonicalName());
}
...
ReplicationLogCleaner.class.toString() will return prefix 'class' to the full
class name, which will make the if checking mostly useless.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619438#comment-13619438 ] Hudson commented on HBASE-7904: --- Integrated in HBase-TRUNK #4005 (See [https://builds.apache.org/job/HBase-TRUNK/4005/]) HBASE-7904 Revert, due to concern over coprocessor exposure (Revision 1463326) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java * /hbase/trunk/pom.xml Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Components: mapreduce Reporter: Ted Yu Priority: Critical Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8238) xref docs missing from site gen
[ https://issues.apache.org/jira/browse/HBASE-8238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619439#comment-13619439 ] Hudson commented on HBASE-8238: --- Integrated in HBase-TRUNK #4005 (See [https://builds.apache.org/job/HBase-TRUNK/4005/]) HBASE-8238 xref docs missing from site gen (Revision 1463321) Result = FAILURE stack : Files : * /hbase/trunk/hbase-assembly/pom.xml * /hbase/trunk/pom.xml xref docs missing from site gen --- Key: HBASE-8238 URL: https://issues.apache.org/jira/browse/HBASE-8238 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.95.0 Attachments: 8238.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8236) Set finalName property in hbase-assembly else basename is hbase-assembly rather than hbase.
[ https://issues.apache.org/jira/browse/HBASE-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619440#comment-13619440 ] Hudson commented on HBASE-8236: --- Integrated in HBase-TRUNK #4005 (See [https://builds.apache.org/job/HBase-TRUNK/4005/]) HBASE-8236 Set finalName property in hbase-assembly else basename is hbase-assembly rather than hbase: ADDENDUM (Revision 1463266) Result = FAILURE stack : Files : * /hbase/trunk/hbase-assembly/pom.xml * /hbase/trunk/pom.xml Set finalName property in hbase-assembly else basename is hbase-assembly rather than hbase. --- Key: HBASE-8236 URL: https://issues.apache.org/jira/browse/HBASE-8236 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.95.0 Attachments: 8236.txt, version.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8133) avoid assign for disabling table regions(OPENING/PENDING_OPEN) in SSH
[ https://issues.apache.org/jira/browse/HBASE-8133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619477#comment-13619477 ] ramkrishna.s.vasudevan commented on HBASE-8133: --- +1. avoid assign for disabling table regions(OPENING/PENDING_OPEN) in SSH - Key: HBASE-8133 URL: https://issues.apache.org/jira/browse/HBASE-8133 Project: HBase Issue Type: Bug Components: Region Assignment Affects Versions: 0.95.0 Reporter: rajeshbabu Assignee: rajeshbabu Fix For: 0.98.0 Attachments: HBASE-8133_2.patch, HBASE-8133.patch Disabling table regions which are in PENDING_OPEN or OPENING on dead server are getting assigned then unassiging. We can avoid this by just mark OFFLINE for the regions,any way znodes of the transitions got deleted as part of am.processServerShutdown(serverName). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8240) CompoundConfiguration should implement Iterable
Ted Yu created HBASE-8240:
-
Summary: CompoundConfiguration should implement Iterable
Key: HBASE-8240
URL: https://issues.apache.org/jira/browse/HBASE-8240
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Here is from hadoop Configuration class:
{code}
public class Configuration implements IterableMap.EntryString,String,
{code}
There're 3 addXX() methods for CompoundConfiguration:
{code}
public CompoundConfiguration add(final Configuration conf) {
public CompoundConfiguration addWritableMap(
final MapImmutableBytesWritable, ImmutableBytesWritable map) {
public CompoundConfiguration addStringMap(final MapString, String map) {
{code}
Parameters to these methods all support iteration.
We can enhance ImmutableConfigMap with the following new method:
{code}
public abstract java.util.Iterator iterator();
{code}
This enhancement would be useful in scenario where a mutable Configuration is
required.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8240) CompoundConfiguration should implement Iterable
[
https://issues.apache.org/jira/browse/HBASE-8240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-8240:
--
Issue Type: Improvement (was: Bug)
CompoundConfiguration should implement Iterable
---
Key: HBASE-8240
URL: https://issues.apache.org/jira/browse/HBASE-8240
Project: HBase
Issue Type: Improvement
Reporter: Ted Yu
Here is from hadoop Configuration class:
{code}
public class Configuration implements IterableMap.EntryString,String,
{code}
There're 3 addXX() methods for CompoundConfiguration:
{code}
public CompoundConfiguration add(final Configuration conf) {
public CompoundConfiguration addWritableMap(
final MapImmutableBytesWritable, ImmutableBytesWritable map) {
public CompoundConfiguration addStringMap(final MapString, String map) {
{code}
Parameters to these methods all support iteration.
We can enhance ImmutableConfigMap with the following new method:
{code}
public abstract java.util.Iterator iterator();
{code}
Then the following method of CompoundConfiguration can be implemented:
{code}
public IteratorMap.EntryString, String iterator() {
{code}
This enhancement would be useful in scenario where a mutable Configuration is
required.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8240) CompoundConfiguration should implement Iterable
[
https://issues.apache.org/jira/browse/HBASE-8240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-8240:
--
Description:
Here is from hadoop Configuration class:
{code}
public class Configuration implements IterableMap.EntryString,String,
{code}
There're 3 addXX() methods for CompoundConfiguration:
{code}
public CompoundConfiguration add(final Configuration conf) {
public CompoundConfiguration addWritableMap(
final MapImmutableBytesWritable, ImmutableBytesWritable map) {
public CompoundConfiguration addStringMap(final MapString, String map) {
{code}
Parameters to these methods all support iteration.
We can enhance ImmutableConfigMap with the following new method:
{code}
public abstract java.util.Iterator iterator();
{code}
Then the following method of CompoundConfiguration can be implemented:
{code}
public IteratorMap.EntryString, String iterator() {
{code}
This enhancement would be useful in scenario where a mutable Configuration is
required.
was:
Here is from hadoop Configuration class:
{code}
public class Configuration implements IterableMap.EntryString,String,
{code}
There're 3 addXX() methods for CompoundConfiguration:
{code}
public CompoundConfiguration add(final Configuration conf) {
public CompoundConfiguration addWritableMap(
final MapImmutableBytesWritable, ImmutableBytesWritable map) {
public CompoundConfiguration addStringMap(final MapString, String map) {
{code}
Parameters to these methods all support iteration.
We can enhance ImmutableConfigMap with the following new method:
{code}
public abstract java.util.Iterator iterator();
{code}
This enhancement would be useful in scenario where a mutable Configuration is
required.
CompoundConfiguration should implement Iterable
---
Key: HBASE-8240
URL: https://issues.apache.org/jira/browse/HBASE-8240
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Here is from hadoop Configuration class:
{code}
public class Configuration implements IterableMap.EntryString,String,
{code}
There're 3 addXX() methods for CompoundConfiguration:
{code}
public CompoundConfiguration add(final Configuration conf) {
public CompoundConfiguration addWritableMap(
final MapImmutableBytesWritable, ImmutableBytesWritable map) {
public CompoundConfiguration addStringMap(final MapString, String map) {
{code}
Parameters to these methods all support iteration.
We can enhance ImmutableConfigMap with the following new method:
{code}
public abstract java.util.Iterator iterator();
{code}
Then the following method of CompoundConfiguration can be implemented:
{code}
public IteratorMap.EntryString, String iterator() {
{code}
This enhancement would be useful in scenario where a mutable Configuration is
required.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8224) Add '-hadoop1' or '-hadoop2' to our version string
[
https://issues.apache.org/jira/browse/HBASE-8224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619483#comment-13619483
]
stack commented on HBASE-8224:
--
Thank you for sharing the pain [~enis].
Here is how I did snapshot upload to maven repo:
{code}
3547 svn revert -R .
3548 svn cleanup
3549 mvn clean org.codehaus.mojo:versions-maven-plugin:1.3.1:set
-DnewVersion=0.95.0-hadoop1-SNAPSHOT
3551 mvn -DskipTests install deploy
3552 svn revert -R .
3553 svn cleanup
3554 mvn clean org.codehaus.mojo:versions-maven-plugin:1.3.1:set
-DnewVersion=0.95.0-hadoop2-SNAPSHOT
3555 mvn -DskipTests -Dhadoop.profile=2.0 install deploy
{code}
It looks ok...
https://repository.apache.org/content/repositories/snapshots/org/apache/hbase/hbase-client/
POMs don't explicitly reference a hadoop version though which is a little
disconcerting.
Add '-hadoop1' or '-hadoop2' to our version string
--
Key: HBASE-8224
URL: https://issues.apache.org/jira/browse/HBASE-8224
Project: HBase
Issue Type: Task
Reporter: stack
Attachments: 8224-adding.classifiers.txt, hbase-8224-proto1.patch
So we can publish both the hadoop1 and the hadoop2 jars to a maven
repository, and so we can publish two packages, one for hadoop1 and one for
hadoop2, given how maven works, our only alternative (to the best of my
knowledge and after consulting others) is by amending the version string to
include hadoop1 or hadoop2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8229) Replication code logs like crazy if a target table cannot be found.
[ https://issues.apache.org/jira/browse/HBASE-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619486#comment-13619486 ] Lars Hofhansl commented on HBASE-8229: -- +1 on some kind of replication status info on the RS UI page. Replication code logs like crazy if a target table cannot be found. --- Key: HBASE-8229 URL: https://issues.apache.org/jira/browse/HBASE-8229 Project: HBase Issue Type: Bug Components: Replication Reporter: Lars Hofhansl Fix For: 0.95.0, 0.98.0, 0.94.7 One of our RS/DN machines ran out of diskspace on the partition to which we write the log files. It turns out we still had a table in our source cluster with REPLICATION_SCOPE=1 that did not have a matching table in the remote cluster. In then logged a long stack trace every 50ms or so, over a few days that filled up our log partition. Since ReplicationSource cannot make any progress in this case anyway, it should probably sleep a bit before retrying (or at least limit the rate at which it spews out these exceptions to the log). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8225) [replication] minor code bug when registering ReplicationLogCleaner
[
https://issues.apache.org/jira/browse/HBASE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8225:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
[replication] minor code bug when registering ReplicationLogCleaner
---
Key: HBASE-8225
URL: https://issues.apache.org/jira/browse/HBASE-8225
Project: HBase
Issue Type: Bug
Components: Replication
Affects Versions: 0.94.6
Reporter: Jerry He
Assignee: Jerry He
Priority: Minor
Fix For: 0.95.0, 0.96.0, 0.94.7
Attachments: HBASE-8255-0.94.patch, HBASE-8255-trunk.patch
We try to register ReplicationLogCleaner by default. This is done by calling
Replication.decorateMasterConfiguration()in the master.
In the current Replication.decorateMasterConfiguration():
...
String plugins = conf.get(HBASE_MASTER_LOGCLEANER_PLUGINS);
if (!plugins.contains(ReplicationLogCleaner.class.toString())) {
conf.set(HBASE_MASTER_LOGCLEANER_PLUGINS,
plugins + , + ReplicationLogCleaner.class.getCanonicalName());
}
...
ReplicationLogCleaner.class.toString() will return prefix 'class' to the full
class name, which will make the if checking mostly useless.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8229) Replication code logs like crazy if a target table cannot be found.
[ https://issues.apache.org/jira/browse/HBASE-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619487#comment-13619487 ] Jieshan Bean commented on HBASE-8229: - Yes, it's really a good idea. Replication code logs like crazy if a target table cannot be found. --- Key: HBASE-8229 URL: https://issues.apache.org/jira/browse/HBASE-8229 Project: HBase Issue Type: Bug Components: Replication Reporter: Lars Hofhansl Fix For: 0.95.0, 0.98.0, 0.94.7 One of our RS/DN machines ran out of diskspace on the partition to which we write the log files. It turns out we still had a table in our source cluster with REPLICATION_SCOPE=1 that did not have a matching table in the remote cluster. In then logged a long stack trace every 50ms or so, over a few days that filled up our log partition. Since ReplicationSource cannot make any progress in this case anyway, it should probably sleep a bit before retrying (or at least limit the rate at which it spews out these exceptions to the log). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-7925) Back port HBASE-6881 into 0.94
[ https://issues.apache.org/jira/browse/HBASE-7925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rajeshbabu resolved HBASE-7925. --- Resolution: Fixed Back port HBASE-6881 into 0.94 -- Key: HBASE-7925 URL: https://issues.apache.org/jira/browse/HBASE-7925 Project: HBase Issue Type: Bug Reporter: rajeshbabu Assignee: rajeshbabu Fix For: 0.94.7 Attachments: HBASE-7925_94.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8127) Region of a disabling or disabled table could be stuck in transition state when RS dies during Master initialization
[
https://issues.apache.org/jira/browse/HBASE-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619492#comment-13619492
]
ramkrishna.s.vasudevan commented on HBASE-8127:
---
[~rajesh23]
I think you and Jeffrey had thought of different approaches and finally decided
on patch_2.
I tried removing the code in AM from your patch and all the testcases including
TestMasterFailOver is passing.
So may be handling for the Disabled and Disabling during master initialization
when in process RIT may not be needed i fee.
The SSH code will any way be triggered for the RS that went down during master
start up(for the RS that went down during start up).
Now there we try to handle the case based on the ZKTable state. That should be
fine. And infact SSH in normal cases also will now handle for
DISABLED/DISABLING tables.
What you feel?
bq.Let us take an example. There are 250 regions in RIT need to process during
initialization and 250 regions of DISABLED table are opened before a server
went down. In that case SSH will take same time as master initialization
becuase in both the cases we need to read from zk.
I can see that you have given an example to substantiate your point. But i do
feel like having same code and doing same job will make our code less readable
and later any bug over here we need to address in 2 places.
Region of a disabling or disabled table could be stuck in transition state
when RS dies during Master initialization
Key: HBASE-8127
URL: https://issues.apache.org/jira/browse/HBASE-8127
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.5
Reporter: Jeffrey Zhong
Assignee: rajeshbabu
Fix For: 0.94.7
Attachments: HBASE-8127_94_2.patch, HBASE-8127_94_3.patch,
HBASE-8127_feedback.patch, HBASE-8127.patch, hbase-8127_v1.patch,
reproduce-hang.patch
The issue happens when a RS dies during a master starts up. After the RS
reports open to the new master instance and dies immediately thereafter, the
RITs of disabling tables(or disabled table) on the died RS will be in RIT
state forever.
I attached a patch to simulate the situation and you can run the following
command to reproduce the issue:
{code}mvn test -PlocalTests
-Dtest=TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS{code}
Basically, we skip regions of a dead server inside
AM.processDeadServersAndRecoverLostRegions as the following code and relies
on SSH to process those skipped regions:
{code}
for (PairHRegionInfo, Result deadRegion : deadServer.getValue()) {
nodes.remove(deadRegion.getFirst().getEncodedName());
}
{code}
While in SSH, we skip regions of disabling(or disabled table) again by
function processDeadRegion. Finally comes to the issue that RITs of
disabling(or disabled table) stuck there forever.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8233) Poke the ZKTable cache and let the system reload cached information
[ https://issues.apache.org/jira/browse/HBASE-8233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8233: -- Summary: Poke the ZKTable cache and let the system reload cached information (was: Poke the cache and let the system reload cached information) Updated the title so that this ZKTable cache is not confused with Block Cache. Poke the ZKTable cache and let the system reload cached information --- Key: HBASE-8233 URL: https://issues.apache.org/jira/browse/HBASE-8233 Project: HBase Issue Type: New Feature Reporter: Jimmy Xiang Sometimes, cached information is stale and we don't have a way to get rid of it except restarting the server. If we can poke the cache and let the system reloads, it will much easier and quicker. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8127) Region of a disabling or disabled table could be stuck in transition state when RS dies during Master initialization
[
https://issues.apache.org/jira/browse/HBASE-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619500#comment-13619500
]
rajeshbabu commented on HBASE-8127:
---
[~ram_krish]
bq. tried removing the code in AM from your patch and all the testcases
including TestMasterFailOver is passing.
Can you run
TestRSKilledWhenMasterInitializing.testMasterFailoverWhenDisablingTableRegionsInRITOnDeadRS
test added in the patch Which will timeout.
bq. And infact SSH in normal cases also will now handle for DISABLED/DISABLING
tables
Its handling only if rit is not null and regions in pending close or closing
state. during master startup we are not able to decide whether the region in
transition or not(regions,rit map dont have entries for the regions). If we
remove null check for rit in SSH,then SSH processing will be slow in normal
case as well(every time unncessarily contact zookeeper to check any znode is
present or not). But the code in AM delete only if node is present.
bq. But i do feel like having same code and doing same job will make our code
less readable and later any bug over here we need to address in 2 places.
We can extract to a method and call the same in 2 places. What do you say Ram?
Region of a disabling or disabled table could be stuck in transition state
when RS dies during Master initialization
Key: HBASE-8127
URL: https://issues.apache.org/jira/browse/HBASE-8127
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.5
Reporter: Jeffrey Zhong
Assignee: rajeshbabu
Fix For: 0.94.7
Attachments: HBASE-8127_94_2.patch, HBASE-8127_94_3.patch,
HBASE-8127_feedback.patch, HBASE-8127.patch, hbase-8127_v1.patch,
reproduce-hang.patch
The issue happens when a RS dies during a master starts up. After the RS
reports open to the new master instance and dies immediately thereafter, the
RITs of disabling tables(or disabled table) on the died RS will be in RIT
state forever.
I attached a patch to simulate the situation and you can run the following
command to reproduce the issue:
{code}mvn test -PlocalTests
-Dtest=TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS{code}
Basically, we skip regions of a dead server inside
AM.processDeadServersAndRecoverLostRegions as the following code and relies
on SSH to process those skipped regions:
{code}
for (PairHRegionInfo, Result deadRegion : deadServer.getValue()) {
nodes.remove(deadRegion.getFirst().getEncodedName());
}
{code}
While in SSH, we skip regions of disabling(or disabled table) again by
function processDeadRegion. Finally comes to the issue that RITs of
disabling(or disabled table) stuck there forever.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8127) Region of a disabling or disabled table could be stuck in transition state when RS dies during Master initialization
[
https://issues.apache.org/jira/browse/HBASE-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619502#comment-13619502
]
ramkrishna.s.vasudevan commented on HBASE-8127:
---
bq.TestRSKilledWhenMasterInitializing.testMasterFailoverWhenDisablingTableRegionsInRITOnDeadRS
test added in the patch Which will timeout.
Region of a disabling or disabled table could be stuck in transition state
when RS dies during Master initialization
Key: HBASE-8127
URL: https://issues.apache.org/jira/browse/HBASE-8127
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.5
Reporter: Jeffrey Zhong
Assignee: rajeshbabu
Fix For: 0.94.7
Attachments: HBASE-8127_94_2.patch, HBASE-8127_94_3.patch,
HBASE-8127_feedback.patch, HBASE-8127.patch, hbase-8127_v1.patch,
reproduce-hang.patch
The issue happens when a RS dies during a master starts up. After the RS
reports open to the new master instance and dies immediately thereafter, the
RITs of disabling tables(or disabled table) on the died RS will be in RIT
state forever.
I attached a patch to simulate the situation and you can run the following
command to reproduce the issue:
{code}mvn test -PlocalTests
-Dtest=TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS{code}
Basically, we skip regions of a dead server inside
AM.processDeadServersAndRecoverLostRegions as the following code and relies
on SSH to process those skipped regions:
{code}
for (PairHRegionInfo, Result deadRegion : deadServer.getValue()) {
nodes.remove(deadRegion.getFirst().getEncodedName());
}
{code}
While in SSH, we skip regions of disabling(or disabled table) again by
function processDeadRegion. Finally comes to the issue that RITs of
disabling(or disabled table) stuck there forever.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-8127) Region of a disabling or disabled table could be stuck in transition state when RS dies during Master initialization
[
https://issues.apache.org/jira/browse/HBASE-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619502#comment-13619502
]
ramkrishna.s.vasudevan edited comment on HBASE-8127 at 4/2/13 4:24 AM:
---
bq.TestRSKilledWhenMasterInitializing.testMasterFailoverWhenDisablingTableRegionsInRITOnDeadRS
test added in the patch Which will timeout.
I ran this. It works mostly. And after you said this once it timedout. (So
may be without AM fix there is some race you mean?).
bq.ff we remove null check for rit in SSH,then SSH processing will be slow in
normal case as well(every time unncessarily contact zookeeper to check any
znode is present or not)
You mean the zkTable node? If you are talking about the zkTable znode, we can
maintain a set locally and just ensure that only for the first time when we get
a new table we hit the ZK, else don't hit.
bq.We can extract to a method and call the same in 2 places
If still we are not convinced, extracting to a method will surely help us.
was (Author: ram_krish):
bq.TestRSKilledWhenMasterInitializing.testMasterFailoverWhenDisablingTableRegionsInRITOnDeadRS
test added in the patch Which will timeout.
Region of a disabling or disabled table could be stuck in transition state
when RS dies during Master initialization
Key: HBASE-8127
URL: https://issues.apache.org/jira/browse/HBASE-8127
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.5
Reporter: Jeffrey Zhong
Assignee: rajeshbabu
Fix For: 0.94.7
Attachments: HBASE-8127_94_2.patch, HBASE-8127_94_3.patch,
HBASE-8127_feedback.patch, HBASE-8127.patch, hbase-8127_v1.patch,
reproduce-hang.patch
The issue happens when a RS dies during a master starts up. After the RS
reports open to the new master instance and dies immediately thereafter, the
RITs of disabling tables(or disabled table) on the died RS will be in RIT
state forever.
I attached a patch to simulate the situation and you can run the following
command to reproduce the issue:
{code}mvn test -PlocalTests
-Dtest=TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS{code}
Basically, we skip regions of a dead server inside
AM.processDeadServersAndRecoverLostRegions as the following code and relies
on SSH to process those skipped regions:
{code}
for (PairHRegionInfo, Result deadRegion : deadServer.getValue()) {
nodes.remove(deadRegion.getFirst().getEncodedName());
}
{code}
While in SSH, we skip regions of disabling(or disabled table) again by
function processDeadRegion. Finally comes to the issue that RITs of
disabling(or disabled table) stuck there forever.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8127) Region of a disabling or disabled table could be stuck in transition state when RS dies during Master initialization
[
https://issues.apache.org/jira/browse/HBASE-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619506#comment-13619506
]
rajeshbabu commented on HBASE-8127:
---
bq.So may be without AM fix there is some race you mean?
yes.
bq. You mean the zkTable node?
No. The RIT node for a region to check whether a region actually in transition
or not.
bq. If still we are not convinced, extracting to a method will surely help us.
I will extract to a method and update the patch.
Region of a disabling or disabled table could be stuck in transition state
when RS dies during Master initialization
Key: HBASE-8127
URL: https://issues.apache.org/jira/browse/HBASE-8127
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.5
Reporter: Jeffrey Zhong
Assignee: rajeshbabu
Fix For: 0.94.7
Attachments: HBASE-8127_94_2.patch, HBASE-8127_94_3.patch,
HBASE-8127_feedback.patch, HBASE-8127.patch, hbase-8127_v1.patch,
reproduce-hang.patch
The issue happens when a RS dies during a master starts up. After the RS
reports open to the new master instance and dies immediately thereafter, the
RITs of disabling tables(or disabled table) on the died RS will be in RIT
state forever.
I attached a patch to simulate the situation and you can run the following
command to reproduce the issue:
{code}mvn test -PlocalTests
-Dtest=TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS{code}
Basically, we skip regions of a dead server inside
AM.processDeadServersAndRecoverLostRegions as the following code and relies
on SSH to process those skipped regions:
{code}
for (PairHRegionInfo, Result deadRegion : deadServer.getValue()) {
nodes.remove(deadRegion.getFirst().getEncodedName());
}
{code}
While in SSH, we skip regions of disabling(or disabled table) again by
function processDeadRegion. Finally comes to the issue that RITs of
disabling(or disabled table) stuck there forever.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8229) Replication code logs like crazy if a target table cannot be found.
[ https://issues.apache.org/jira/browse/HBASE-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8229: - Attachment: 8229-0.94.txt Here's a simple 0.94 fix. While looking at the code I also realized that there is no way to get out of this situation in the source cluster. Removing REPLICATION_SCOPE, disabling, or even dropping the table will not end this endless loop. Replication will never make any progress until the RS is restarted or the table is added to the peer cluster. Replication code logs like crazy if a target table cannot be found. --- Key: HBASE-8229 URL: https://issues.apache.org/jira/browse/HBASE-8229 Project: HBase Issue Type: Bug Components: Replication Reporter: Lars Hofhansl Fix For: 0.95.0, 0.98.0, 0.94.7 Attachments: 8229-0.94.txt One of our RS/DN machines ran out of diskspace on the partition to which we write the log files. It turns out we still had a table in our source cluster with REPLICATION_SCOPE=1 that did not have a matching table in the remote cluster. In then logged a long stack trace every 50ms or so, over a few days that filled up our log partition. Since ReplicationSource cannot make any progress in this case anyway, it should probably sleep a bit before retrying (or at least limit the rate at which it spews out these exceptions to the log). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6651) Improve thread safety of HTablePool
[ https://issues.apache.org/jira/browse/HBASE-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hiroshi Ikeda updated HBASE-6651: - Attachment: HBASE-6651-V13.patch Patch v13 from review board. Improve thread safety of HTablePool --- Key: HBASE-6651 URL: https://issues.apache.org/jira/browse/HBASE-6651 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.94.1 Reporter: Hiroshi Ikeda Assignee: Hiroshi Ikeda Fix For: 0.95.0 Attachments: HBASE-6651.patch, HBASE-6651-V10.patch, HBASE-6651-V11.patch, HBASE-6651-V12.patch, HBASE-6651-V13.patch, HBASE-6651-V2.patch, HBASE-6651-V3.patch, HBASE-6651-V4.patch, HBASE-6651-V5.patch, HBASE-6651-V6.patch, HBASE-6651-V7.patch, HBASE-6651-V8.patch, HBASE-6651-V9.patch, sample.zip, sample.zip, sharedmap_for_hbaseclient.zip There are some operations in HTablePool accessing PoolMap in multiple places without any explicit synchronization. For example HTablePool.closeTablePool() calls PoolMap.values(), and calls PoolMap.remove(). If other threads add new instances to the pool in the middle of the calls, the newly added instances might be dropped. (HTablePool.closeTablePool() also has another problem that calling it by multiple threads causes accessing HTable by multiple threads.) Moreover, PoolMap is not thread safe for the same reason. For example PoolMap.put() calles ConcurrentMap.get() and calles ConcurrentMap.put(). If other threads add a new instance to the concurent map in the middle of the calls, the new instance might be dropped. And also implementations of Pool have the same problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
