[jira] [Commented] (HBASE-2611) Handle RS that fails while processing the failure of another one
[ https://issues.apache.org/jira/browse/HBASE-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165085#comment-13165085 ] Chris Trezzo commented on HBASE-2611: - @J-D LarsH and I were talking about another approach to region server replication hlog queue failover yesterday, and I wanted to get some feedback on it. Currently when handling a nodeDeleted event, the live region servers only attempt to failover the node corresponding to the event. The nodeDeleted event is only fired once, so to protect ourselves from orphaning the znode state of the failed region server in a cascading failure scenario, we move the state to the znode of the region server that is performing the failover. Since we don't have an atomic way to move this state, it gets a little tricky. Instead of this approach, we could have the region server attempt to failover all failed region servers every time it receives a nodeDeleted event. For example, the nodeDeleted method could go something like this: refresh the region server list, get the list of region servers in the replication znode structure, attempt to lock and failover any region server listed in the replication znode structure that is not currently alive. The same race to lock the region server znode will occur. Only one region server will get the lock and handle the failover. Each NodeFailoverWorker that gets started could simply operate on the original dead region server znode structure. If the region server fails while preforming the failover, then both the region servers will get picked up by another region server when the nodeDeleted event for the second failure is fired. Locks would have to be ephemeral nodes to prevent permanent locking of a region server when the failover region server dies. Once the replication hlog queues are successfully replicated, the znode for the dead region server can be deleted. On the cons side, this approach makes the handling of a nodeDeleted event a heavier weight operation. On the pros side, it makes the failover code much simpler because we no longer have to worry about moving the region server znode state around in zookeeper. Thoughts always appreciated. Thanks, Chris Handle RS that fails while processing the failure of another one Key: HBASE-2611 URL: https://issues.apache.org/jira/browse/HBASE-2611 Project: HBase Issue Type: Sub-task Components: replication Reporter: Jean-Daniel Cryans Assignee: Jean-Daniel Cryans HBASE-2223 doesn't manage region servers that fail while doing the transfer of HLogs queues from other region servers that failed. Devise a reliable way to do it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4908) HBase cluster test tool (port from 0.89-fb)
[ https://issues.apache.org/jira/browse/HBASE-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165091#comment-13165091 ] Hudson commented on HBASE-4908: --- Integrated in HBase-TRUNK #2526 (See [https://builds.apache.org/job/HBase-TRUNK/2526/]) [jira] [HBASE-4908] HBase cluster test tool (port from 0.89-fb) Summary: Porting one of our HBase cluster test tools (a single-process multi-threaded load generator and verifier) from 0.89-fb to trunk. I cleaned up the code a bit compared to what's in 0.89-fb, and discovered that it has some features that I have not tried yet (some kind of a kill test, and some way to run HBase as multiple processes on one machine). The main utility of this piece of code for us has been the HBaseClusterTest command-line tool (called HBaseTest in 0.89-fb), which we usually invoke as a load test in our five-node dev cluster testing, e.g.: hbase org.apache.hadoop.hbase.util.LoadTestTool -write 50:100:20 -tn loadtest4 -read 100:10 -zk zk_quorum_node -bloom ROWCOL -compression LZO -key_window 5 -max_read_errors 1 -num_keys 100 -start_key 0 Test Plan: Run this on a dev cluster. Run all unit tests. Reviewers: stack, Karthik, Kannan, nspiegelberg, JIRA Reviewed By: nspiegelberg CC: stack, nspiegelberg, mbautin, Karthik Differential Revision: 549 nspiegelberg : Files : * /hbase/trunk/pom.xml * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/EmptyWatcher.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogReader.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/AbstractHBaseTool.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/Bytes.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HMerge.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/Keying.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/RegionSplitter.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKConfig.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/EmptyWatcher.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionSplitPolicy.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/LoadTestKVGenerator.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/MultiThreadedAction.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/MultiThreadedReader.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/MultiThreadedWriter.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/ProcessBasedLocalHBaseCluster.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/RestartMetaTest.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestBytes.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestLoadTestKVGenerator.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestMiniClusterLoadParallel.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestMiniClusterLoadSequential.java HBase cluster test tool (port from 0.89-fb) --- Key: HBASE-4908 URL: https://issues.apache.org/jira/browse/HBASE-4908 Project: HBase Issue Type: Sub-task Reporter: Mikhail Bautin Assignee: Mikhail Bautin Fix For: 0.94.0 Attachments: 0001-HBase-cluster-test-tool.patch, 0002-HBase-cluster-test-tool.patch, 0003-HBase-cluster-test-tool.patch, D549.1.patch, D549.2.patch, D549.3.patch, D549.4.patch, D549.5.patch, D549.6.patch, D549.7.patch, D549.8.patch, D549.9.patch Porting one of our
[jira] [Commented] (HBASE-4610) Port HBASE-3380 (Master failover can split logs of live servers) to 92/trunk (definitely bring in config params, decide if we need to do more to fix the bug)
[ https://issues.apache.org/jira/browse/HBASE-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165089#comment-13165089 ] Hudson commented on HBASE-4610: --- Integrated in HBase-TRUNK #2526 (See [https://builds.apache.org/job/HBase-TRUNK/2526/]) HBASE-4610 Addendum removes break statement HBASE-4610 Port HBASE-3380 (Master failover can split logs of live servers) to 92/trunk tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java tedyu : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java Port HBASE-3380 (Master failover can split logs of live servers) to 92/trunk (definitely bring in config params, decide if we need to do more to fix the bug) - Key: HBASE-4610 URL: https://issues.apache.org/jira/browse/HBASE-4610 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.0, 0.94.0 Reporter: Jonathan Gray Assignee: Jonathan Gray Fix For: 0.92.0, 0.94.0 Attachments: 4610.txt Over in HBASE-3380 we were having some TestMasterFailover flakiness. We added some more config parameters to better control the master startup loop where it waits for RS to heartbeat in. We had thought at the time that 92 would have a different solution but it is still relying on heartbeats to learn about RSs. For now, we should definitely bring these config params into 92/trunk. Otherwise this is an incompatible regression and adding these will also make things like what was just reported over in HBASE-4603 trivial to fix in an optimal way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3380) Master failover can split logs of live servers
[
https://issues.apache.org/jira/browse/HBASE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165092#comment-13165092
]
Hudson commented on HBASE-3380:
---
Integrated in HBase-TRUNK #2526 (See
[https://builds.apache.org/job/HBase-TRUNK/2526/])
HBASE-4610 Port HBASE-3380 (Master failover can split logs of live
servers) to 92/trunk
tedyu :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java
Master failover can split logs of live servers
--
Key: HBASE-3380
URL: https://issues.apache.org/jira/browse/HBASE-3380
Project: HBase
Issue Type: Bug
Reporter: Jean-Daniel Cryans
Assignee: Jonathan Gray
Priority: Blocker
Fix For: 0.90.0
Attachments: HBASE-3380-v1.patch, HBASE-3380-v2.patch
The reason why TestMasterFailover fails is that when it does the master
failover, the new master doesn't wait long enough for all region servers to
checkin so it goes ahead and split logs... which doesn't work because of the
way lease timeouts work:
{noformat}
2010-12-21 07:30:36,977 DEBUG [Master:0;vesta.apache.org:33170]
wal.HLogSplitter(256): Splitting hlog 1 of 1:
hdfs://localhost:49187/user/hudson/.logs/vesta.apache.org,38743,1292916616340/vesta.apache.org%3A38743.1292916617204,
length=0
2010-12-21 07:30:36,977 DEBUG [WriterThread-1]
wal.HLogSplitter$WriterThread(619): Writer thread
Thread[WriterThread-1,5,main]: starting
2010-12-21 07:30:36,977 DEBUG [WriterThread-2]
wal.HLogSplitter$WriterThread(619): Writer thread
Thread[WriterThread-2,5,main]: starting
2010-12-21 07:30:36,977 INFO [Master:0;vesta.apache.org:33170]
util.FSUtils(625): Recovering file
hdfs://localhost:49187/user/hudson/.logs/vesta.apache.org,38743,1292916616340/vesta.apache.org%3A38743.1292916617204
2010-12-21 07:30:36,979 WARN [IPC Server handler 8 on 49187]
namenode.FSNamesystem(1122): DIR* NameSystem.startFile:
failed to create file
/user/hudson/.logs/vesta.apache.org,38743,1292916616340/vesta.apache.org%3A38743.1292916617204
for
DFSClient_hb_m_vesta.apache.org:33170_1292916630791 on client 127.0.0.1,
because this file is already being created by
DFSClient_hb_rs_vesta.apache.org,38743,1292916616340_1292916617166 on
127.0.0.1
...
2010-12-21 07:33:44,332 WARN [Master:0;vesta.apache.org:33170]
util.FSUtils(644): Waited 187354ms for lease recovery on
hdfs://localhost:49187/user/hudson/.logs/vesta.apache.org,38743,1292916616340/vesta.apache.org%3A38743.1292916617204:
org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to
create file
/user/hudson/.logs/vesta.apache.org,38743,1292916616340/vesta.apache.org%3A38743.1292916617204
for DFSClient_hb_m_vesta.apache.org:33170_1292916630791 on client 127.0.0.1,
because this file is already
being created by
DFSClient_hb_rs_vesta.apache.org,38743,1292916616340_1292916617166 on
127.0.0.1
{noformat}
I think that we should always check in ZK the number of live region servers
before waiting for them to check in, this way we know how many we should
expect during failover. There's also a case where we still want to timeout,
since RS can die during that time, but we should wait a bit longer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4976) Add compaction/flush queue size metrics mistakenly removed by HFile v2
[ https://issues.apache.org/jira/browse/HBASE-4976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165090#comment-13165090 ] Hudson commented on HBASE-4976: --- Integrated in HBase-TRUNK #2526 (See [https://builds.apache.org/job/HBase-TRUNK/2526/]) HBASE-4976 Add compaction/flush queue size metrics mistakenly removed by HFile v2 stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java Add compaction/flush queue size metrics mistakenly removed by HFile v2 -- Key: HBASE-4976 URL: https://issues.apache.org/jira/browse/HBASE-4976 Project: HBase Issue Type: Bug Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Blocker Fix For: 0.92.0 Attachments: D645.1.patch Upping priority, and putting it against 0.92 since J-D fingered it as blocker. Which metrics in particular are missing? Hard to patch? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165118#comment-13165118 ] jirapos...@reviews.apache.org commented on HBASE-4120: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1421/ --- (Updated 2011-12-08 09:41:50.209106) Review request for hbase. Changes --- The coprocessor version of this patch. Summary --- Patch used for table priority alone,In this patch, not only tables can have different priorities but also the different actions like get,scan,put and delete can have priorities. This addresses bug HBase-4120. https://issues.apache.org/jira/browse/HBase-4120 Diffs (updated) - http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Delete.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Get.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiPut.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Mutation.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Operation.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/PriorityFunction.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/PriorityHBaseServer.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/PriorityJobQueue.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/QosRegionObserver.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/ipc/TestActionPriority.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/ipc/TestPriorityJobQueue.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/ipc/TestTablePriority.java PRE-CREATION Diff: https://reviews.apache.org/r/1421/diff Testing --- Tested with test cases in TestCase_For_TablePriority_trunk_v1.patch please apply the patch of HBASE-4181 first,in some circumstances this bug will affect the performance of client. Thanks, Jia isolation and allocation Key: HBASE-4120 URL: https://issues.apache.org/jira/browse/HBASE-4120 Project: HBase Issue Type: New Feature Components: master, regionserver Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, Design_document_for_HBase_isolation_and_allocation_Revised.pdf, HBase_isolation_and_allocation_user_guide.pdf, Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v8.patch, TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables. When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished. After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group. In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables Different groups can have different
[jira] [Commented] (HBASE-4979) Setting KEEP_DELETE_CELLS fails in shell
[ https://issues.apache.org/jira/browse/HBASE-4979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165193#comment-13165193 ] Hudson commented on HBASE-4979: --- Integrated in HBase-TRUNK #2527 (See [https://builds.apache.org/job/HBase-TRUNK/2527/]) HBASE-4979 Setting KEEP_DELETE_CELLS fails in shell (Lars H) larsh : Files : * /hbase/trunk/src/main/ruby/hbase/admin.rb Setting KEEP_DELETE_CELLS fails in shell Key: HBASE-4979 URL: https://issues.apache.org/jira/browse/HBASE-4979 Project: HBase Issue Type: Sub-task Components: shell Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4979.txt admin.rb uses wrong method on HColumnDescriptor to enable keeping of deleted cells. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4973) On failure, HBaseAdmin sleeps one time too many
[
https://issues.apache.org/jira/browse/HBASE-4973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165246#comment-13165246
]
nkeywal commented on HBASE-4973:
I believe it can be committed.
On failure, HBaseAdmin sleeps one time too many
---
Key: HBASE-4973
URL: https://issues.apache.org/jira/browse/HBASE-4973
Project: HBase
Issue Type: Improvement
Components: client
Affects Versions: 0.94.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
Attachments: 4973_HBaseAdmin.patch
In this code last sleep is useless as we're not retrying. This can slow down
failure scenarios by a few seconds (up to 32 second).
{noformat}
public HBaseAdmin(Configuration c)
throws MasterNotRunningException, ZooKeeperConnectionException {
this.conf = HBaseConfiguration.create(c);
this.connection = HConnectionManager.getConnection(this.conf);
this.pause = this.conf.getLong(hbase.client.pause, 1000);
this.numRetries = this.conf.getInt(hbase.client.retries.number, 10);
this.retryLongerMultiplier = this.conf.getInt(
hbase.client.retries.longer.multiplier, 10);
int tries = 0;
for (; tries numRetries; ++tries) {
try {
this.connection.getMaster();
break;
} catch (MasterNotRunningException mnre) {
HConnectionManager.deleteStaleConnection(this.connection);
this.connection = HConnectionManager.getConnection(this.conf);
} catch (UndeclaredThrowableException ute) {
HConnectionManager.deleteStaleConnection(this.connection);
this.connection = HConnectionManager.getConnection(this.conf);
}
try { // Sleep
Thread.sleep(getPauseTime(tries));
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
// we should delete connection between client and zookeeper
HConnectionManager.deleteStaleConnection(this.connection);
throw new MasterNotRunningException(Interrupted);
}
}
if (tries = numRetries) {
// we should delete connection between client and zookeeper
HConnectionManager.deleteStaleConnection(this.connection);
throw new MasterNotRunningException(Retried + numRetries + times);
}
}
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4965) Monitor the open file descriptors and the threads counters during the unit tests
[ https://issues.apache.org/jira/browse/HBASE-4965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165247#comment-13165247 ] nkeywal commented on HBASE-4965: I summarize the main infos: - it seems that hadoop-qa is configured with 1024 file descriptors - we have some leaks - the patch itself could be committed imho. Monitor the open file descriptors and the threads counters during the unit tests Key: HBASE-4965 URL: https://issues.apache.org/jira/browse/HBASE-4965 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 4965_all.patch, ResourceChecker.java, ResourceCheckerJUnitRule.java We're seeing a lot of issues with hadoop-qa related to threads or file descriptors. Monitoring these counters would ease the analysis. Note as well that - if we want to execute the tests in the same jvm (because the test is small or because we want to share the cluster) we can't afford to leak too many resources - if the tests leak, it's more difficult to detect a leak in the software itself. I attach piece of code that I used. It requires two lines in a unit test class to: - before every test, count the threads and the open file descriptor - after every test, compare with the previous value. I ran it on some tests; we have for example: - client.TestMultiParallel#testBatchWithManyColsInOneRowGetAndPut: 232 threads (was 231), 390 file descriptors (was 390). = TestMultiParallel uses 232 threads! - client.TestMultipleTimestamps#testWithColumnDeletes: 152 threads (was 151), 283 file descriptors (was 282). - client.TestAdmin#testCheckHBaseAvailableClosesConnection: 477 threads (was 294), 815 file descriptors (was 461) - client.TestMetaMigrationRemovingHTD#testMetaMigration: 149 threads (was 148), 310 file descriptors (was 307). It's not always leaks, we can expect some pooling effects. But still... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4983) index.html - correcting BigTable link, changing intro line, adding one feature
index.html - correcting BigTable link, changing intro line, adding one feature -- Key: HBASE-4983 URL: https://issues.apache.org/jira/browse/HBASE-4983 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor index.html * correcting BigTable link (existing one was actually a 404!) * Changed top-level description to Think of it as a distributed scalable Big Data store. * Added a feature linear and modular scalability -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4983) index.html - correcting BigTable link, changing intro line, adding one feature
[ https://issues.apache.org/jira/browse/HBASE-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4983: - Status: Patch Available (was: Open) index.html - correcting BigTable link, changing intro line, adding one feature -- Key: HBASE-4983 URL: https://issues.apache.org/jira/browse/HBASE-4983 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: index_HBASE_4983.xml.patch index.html * correcting BigTable link (existing one was actually a 404!) * Changed top-level description to Think of it as a distributed scalable Big Data store. * Added a feature linear and modular scalability -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4983) index.html - correcting BigTable link, changing intro line, adding one feature
[ https://issues.apache.org/jira/browse/HBASE-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4983: - Attachment: index_HBASE_4983.xml.patch index.html - correcting BigTable link, changing intro line, adding one feature -- Key: HBASE-4983 URL: https://issues.apache.org/jira/browse/HBASE-4983 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: index_HBASE_4983.xml.patch index.html * correcting BigTable link (existing one was actually a 404!) * Changed top-level description to Think of it as a distributed scalable Big Data store. * Added a feature linear and modular scalability -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4983) index.html - correcting BigTable link, changing intro line, adding one feature
[ https://issues.apache.org/jira/browse/HBASE-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4983: - Resolution: Fixed Status: Resolved (was: Patch Available) index.html - correcting BigTable link, changing intro line, adding one feature -- Key: HBASE-4983 URL: https://issues.apache.org/jira/browse/HBASE-4983 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: index_HBASE_4983.xml.patch index.html * correcting BigTable link (existing one was actually a 404!) * Changed top-level description to Think of it as a distributed scalable Big Data store. * Added a feature linear and modular scalability -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4984) index.html - adding a link to architecture overview in more information section.
[ https://issues.apache.org/jira/browse/HBASE-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4984: - Status: Patch Available (was: Open) index.html - adding a link to architecture overview in more information section. -- Key: HBASE-4984 URL: https://issues.apache.org/jira/browse/HBASE-4984 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: index_hbase_4984.xml.patch index.html * adding link to architecture overview in the book from index.html in the More Information? section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4981) add raw scan support to shell
[
https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165364#comment-13165364
]
Lars Hofhansl commented on HBASE-4981:
--
With a family delete marker:
{noformat}
hbase(main):005:0 scan 'x1', {RAW=true, VERSIONS=10}
ROW COLUMN+CELL
r2 column=f:, timestamp=1323323616226, type=DeleteFamily
r2 column=f:c, timestamp=1323323617759, value=v3
r2 column=f:c, timestamp=1323323616226, value=v2
r2 column=f:c, timestamp=1323323614496, value=v1
2 row(s) in 0.0500 seconds
{noformat}
Let me know what guys think, I think this is simple change and ready to go in.
add raw scan support to shell
-
Key: HBASE-4981
URL: https://issues.apache.org/jira/browse/HBASE-4981
Project: HBase
Issue Type: Sub-task
Components: shell
Affects Versions: 0.94.0
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.94.0
Attachments: 4981-v1.txt
Parent adds raw scan support to include delete markers and deleted rows in
scan results. Would be nice it that would available in the shell to see
exactly what exists in a table.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4682: - Status: Open (was: Patch Available) Support deleted rows using Import/Export Key: HBASE-4682 URL: https://issues.apache.org/jira/browse/HBASE-4682 Project: HBase Issue Type: Sub-task Components: mapreduce Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4682-v1.txt Parent allows keeping deleted rows around. Would be nice if those could be exported and imported as well. All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4984) index.html - adding a link to architecture overview in more information section.
[ https://issues.apache.org/jira/browse/HBASE-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4984: - Attachment: index_hbase_4984.xml.patch index.html - adding a link to architecture overview in more information section. -- Key: HBASE-4984 URL: https://issues.apache.org/jira/browse/HBASE-4984 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: index_hbase_4984.xml.patch index.html * adding link to architecture overview in the book from index.html in the More Information? section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4984) index.html - adding a link to architecture overview in more information section.
index.html - adding a link to architecture overview in more information section. -- Key: HBASE-4984 URL: https://issues.apache.org/jira/browse/HBASE-4984 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: index_hbase_4984.xml.patch index.html * adding link to architecture overview in the book from index.html in the More Information? section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4984) index.html - adding a link to architecture overview in more information section.
[ https://issues.apache.org/jira/browse/HBASE-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4984: - Resolution: Fixed Status: Resolved (was: Patch Available) index.html - adding a link to architecture overview in more information section. -- Key: HBASE-4984 URL: https://issues.apache.org/jira/browse/HBASE-4984 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: index_hbase_4984.xml.patch index.html * adding link to architecture overview in the book from index.html in the More Information? section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4969) tautology in HRegionInfo.readFields
[
https://issues.apache.org/jira/browse/HBASE-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HBASE-4969:
---
Attachment: HBASE-4969.D669.1.patch
khemani requested code review of HBASE-4969 [jira] tautology in
HRegionInfo.readFields.
Reviewers: nspiegelberg, dhruba, JIRA
as in title
TEST PLAN
minor change.
REVISION DETAIL
https://reviews.facebook.net/D669
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/1479/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
tautology in HRegionInfo.readFields
---
Key: HBASE-4969
URL: https://issues.apache.org/jira/browse/HBASE-4969
Project: HBase
Issue Type: Bug
Reporter: Prakash Khemani
Assignee: Prakash Khemani
Attachments: HBASE-4969.D669.1.patch
In HRegionInfo.readFields() the following looks wrong to me
} else if (getVersion() == VERSION) {
it is always true.
Should it have been
} else if (getVersion() == version) {
version is a local variable where the deserialized-version is stored.
(I am struggling with another issue where after applying some patches -
including HBASE-4388 Second start after migration from 90 to trunk crashes
my version of hbase-92 HRegionInfo.readFields() tries to find HTD in
HRegionInfo and fails)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4983) index.html - correcting BigTable link, changing intro line, adding one feature
[ https://issues.apache.org/jira/browse/HBASE-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165403#comment-13165403 ] Hudson commented on HBASE-4983: --- Integrated in HBase-TRUNK #2529 (See [https://builds.apache.org/job/HBase-TRUNK/2529/]) hbase-4983. index.html: one correction, one change, one addition. index.html - correcting BigTable link, changing intro line, adding one feature -- Key: HBASE-4983 URL: https://issues.apache.org/jira/browse/HBASE-4983 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: index_HBASE_4983.xml.patch index.html * correcting BigTable link (existing one was actually a 404!) * Changed top-level description to Think of it as a distributed scalable Big Data store. * Added a feature linear and modular scalability -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4980) Null pointer exception in HBaseClient receiveResponse
[
https://issues.apache.org/jira/browse/HBASE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shrijeet Paliwal updated HBASE-4980:
Attachment: 0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
Null pointer exception in HBaseClient receiveResponse
-
Key: HBASE-4980
URL: https://issues.apache.org/jira/browse/HBASE-4980
Project: HBase
Issue Type: Bug
Components: client
Affects Versions: 0.92.0
Reporter: Shrijeet Paliwal
Labels: newbie
Attachments:
0001-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
0002-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
Relevant Stack trace:
2011-11-30 13:10:26,557 [IPC Client (47) connection to
xx.xx.xx/172.22.4.68:60020 from an unknown user] WARN
org.apache.hadoop.ipc.HBaseClient - Unexpected exception receiving call
responses
java.lang.NullPointerException
-at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:583)
-at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:511)
{code}
if (LOG.isDebugEnabled())
LOG.debug(getName() + got value # + id);
Call call = calls.remove(id);
// Read the flag byte
byte flag = in.readByte();
boolean isError = ResponseFlag.isError(flag);
if (ResponseFlag.isLength(flag)) {
// Currently length if present is unused.
in.readInt();
}
int state = in.readInt(); // Read the state. Currently unused.
if (isError) {
//noinspection ThrowableInstanceNeverThrown
call.setException(new RemoteException( WritableUtils.readString(in),
WritableUtils.readString(in)));
} else {
{code}
This line {code}Call call = calls.remove(id);{code} may return a null
'call'. It is so because if you have rpc timeout enable, we proactively clean
up other calls which have expired their lifetime along with the call for
which socket timeout exception happend.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4980) Null pointer exception in HBaseClient receiveResponse
[
https://issues.apache.org/jira/browse/HBASE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165406#comment-13165406
]
Shrijeet Paliwal commented on HBASE-4980:
-
Done attaching, should I click cancel patch and then click submit patch again?
Null pointer exception in HBaseClient receiveResponse
-
Key: HBASE-4980
URL: https://issues.apache.org/jira/browse/HBASE-4980
Project: HBase
Issue Type: Bug
Components: client
Affects Versions: 0.92.0
Reporter: Shrijeet Paliwal
Labels: newbie
Attachments:
0001-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
0002-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
Relevant Stack trace:
2011-11-30 13:10:26,557 [IPC Client (47) connection to
xx.xx.xx/172.22.4.68:60020 from an unknown user] WARN
org.apache.hadoop.ipc.HBaseClient - Unexpected exception receiving call
responses
java.lang.NullPointerException
-at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:583)
-at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:511)
{code}
if (LOG.isDebugEnabled())
LOG.debug(getName() + got value # + id);
Call call = calls.remove(id);
// Read the flag byte
byte flag = in.readByte();
boolean isError = ResponseFlag.isError(flag);
if (ResponseFlag.isLength(flag)) {
// Currently length if present is unused.
in.readInt();
}
int state = in.readInt(); // Read the state. Currently unused.
if (isError) {
//noinspection ThrowableInstanceNeverThrown
call.setException(new RemoteException( WritableUtils.readString(in),
WritableUtils.readString(in)));
} else {
{code}
This line {code}Call call = calls.remove(id);{code} may return a null
'call'. It is so because if you have rpc timeout enable, we proactively clean
up other calls which have expired their lifetime along with the call for
which socket timeout exception happend.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4980) Null pointer exception in HBaseClient receiveResponse
[
https://issues.apache.org/jira/browse/HBASE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165413#comment-13165413
]
stack commented on HBASE-4980:
--
If you see nothing in next hour, yes.
Null pointer exception in HBaseClient receiveResponse
-
Key: HBASE-4980
URL: https://issues.apache.org/jira/browse/HBASE-4980
Project: HBase
Issue Type: Bug
Components: client
Affects Versions: 0.92.0
Reporter: Shrijeet Paliwal
Labels: newbie
Attachments:
0001-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
0002-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
Relevant Stack trace:
2011-11-30 13:10:26,557 [IPC Client (47) connection to
xx.xx.xx/172.22.4.68:60020 from an unknown user] WARN
org.apache.hadoop.ipc.HBaseClient - Unexpected exception receiving call
responses
java.lang.NullPointerException
-at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:583)
-at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:511)
{code}
if (LOG.isDebugEnabled())
LOG.debug(getName() + got value # + id);
Call call = calls.remove(id);
// Read the flag byte
byte flag = in.readByte();
boolean isError = ResponseFlag.isError(flag);
if (ResponseFlag.isLength(flag)) {
// Currently length if present is unused.
in.readInt();
}
int state = in.readInt(); // Read the state. Currently unused.
if (isError) {
//noinspection ThrowableInstanceNeverThrown
call.setException(new RemoteException( WritableUtils.readString(in),
WritableUtils.readString(in)));
} else {
{code}
This line {code}Call call = calls.remove(id);{code} may return a null
'call'. It is so because if you have rpc timeout enable, we proactively clean
up other calls which have expired their lifetime along with the call for
which socket timeout exception happend.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4969) tautology in HRegionInfo.readFields
[
https://issues.apache.org/jira/browse/HBASE-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165421#comment-13165421
]
Phabricator commented on HBASE-4969:
stack has accepted the revision HBASE-4969 [jira] tautology in
HRegionInfo.readFields.
+1
INLINE COMMENTS
src/main/java/org/apache/hadoop/hbase/HRegionInfo.java:740 Your fix looks
right Prakash.
REVISION DETAIL
https://reviews.facebook.net/D669
tautology in HRegionInfo.readFields
---
Key: HBASE-4969
URL: https://issues.apache.org/jira/browse/HBASE-4969
Project: HBase
Issue Type: Bug
Reporter: Prakash Khemani
Assignee: Prakash Khemani
Attachments: HBASE-4969.D669.1.patch
In HRegionInfo.readFields() the following looks wrong to me
} else if (getVersion() == VERSION) {
it is always true.
Should it have been
} else if (getVersion() == version) {
version is a local variable where the deserialized-version is stored.
(I am struggling with another issue where after applying some patches -
including HBASE-4388 Second start after migration from 90 to trunk crashes
my version of hbase-92 HRegionInfo.readFields() tries to find HTD in
HRegionInfo and fails)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4655) Document architecture of backups
[ https://issues.apache.org/jira/browse/HBASE-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165436#comment-13165436 ] Karthik Ranganathan commented on HBASE-4655: @Doug: list all the regions, for each region, ask the RS hosting it for a list of HFiles There is already an API to get a list of regions and the regionservers hosting them. And we added a new API to the RS to list the HFiles for the regions it hosts. The strategy is great, but it will generate a flurry of (warranted) questions on how the average person does it. True - but this task is only to make sure the document is easy to read and understand by an average user. We can definitely add more details if needed, but that would risk confusing people. I will definitely incorporate the other suggestions (confusing names, etc). The rest of the tasks deal with giving a way for the average users to do backups by running/cron-ing a command and not have to deal with the internals of how it works. Document architecture of backups Key: HBASE-4655 URL: https://issues.apache.org/jira/browse/HBASE-4655 Project: HBase Issue Type: Sub-task Components: documentation, regionserver Reporter: Karthik Ranganathan Assignee: Karthik Ranganathan Attachments: HBase Backups Architecture.docx Basic idea behind the backup architecture for HBase -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4980) Null pointer exception in HBaseClient receiveResponse
[
https://issues.apache.org/jira/browse/HBASE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165448#comment-13165448
]
Hadoop QA commented on HBASE-4980:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12506636/0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
-1 javadoc. The javadoc tool appears to have generated -160 warning
messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 74 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/475//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/475//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/475//console
This message is automatically generated.
Null pointer exception in HBaseClient receiveResponse
-
Key: HBASE-4980
URL: https://issues.apache.org/jira/browse/HBASE-4980
Project: HBase
Issue Type: Bug
Components: client
Affects Versions: 0.92.0
Reporter: Shrijeet Paliwal
Labels: newbie
Attachments:
0001-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
0002-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
Relevant Stack trace:
2011-11-30 13:10:26,557 [IPC Client (47) connection to
xx.xx.xx/172.22.4.68:60020 from an unknown user] WARN
org.apache.hadoop.ipc.HBaseClient - Unexpected exception receiving call
responses
java.lang.NullPointerException
-at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:583)
-at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:511)
{code}
if (LOG.isDebugEnabled())
LOG.debug(getName() + got value # + id);
Call call = calls.remove(id);
// Read the flag byte
byte flag = in.readByte();
boolean isError = ResponseFlag.isError(flag);
if (ResponseFlag.isLength(flag)) {
// Currently length if present is unused.
in.readInt();
}
int state = in.readInt(); // Read the state. Currently unused.
if (isError) {
//noinspection ThrowableInstanceNeverThrown
call.setException(new RemoteException( WritableUtils.readString(in),
WritableUtils.readString(in)));
} else {
{code}
This line {code}Call call = calls.remove(id);{code} may return a null
'call'. It is so because if you have rpc timeout enable, we proactively clean
up other calls which have expired their lifetime along with the call for
which socket timeout exception happend.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4969) tautology in HRegionInfo.readFields
[
https://issues.apache.org/jira/browse/HBASE-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165451#comment-13165451
]
Phabricator commented on HBASE-4969:
stack has commented on the revision HBASE-4969 [jira] tautology in
HRegionInfo.readFields.
How I fix this Prakash?
h-24-30:trunk stack$ arc amend --revision D669
Usage Exception: Failed to load library arclib at location .arc_jira_lib.
Please check the phutil_libraries setting in your .arcconfig file. Refer to
page http://phabricator.com/docs/arcanist/article/Setting_Up_.arcconfig.html
for more info.
I'm trying to do what Mikhail asked me do,
REVISION DETAIL
https://reviews.facebook.net/D669
tautology in HRegionInfo.readFields
---
Key: HBASE-4969
URL: https://issues.apache.org/jira/browse/HBASE-4969
Project: HBase
Issue Type: Bug
Reporter: Prakash Khemani
Assignee: Prakash Khemani
Attachments: HBASE-4969.D669.1.patch
In HRegionInfo.readFields() the following looks wrong to me
} else if (getVersion() == VERSION) {
it is always true.
Should it have been
} else if (getVersion() == version) {
version is a local variable where the deserialized-version is stored.
(I am struggling with another issue where after applying some patches -
including HBASE-4388 Second start after migration from 90 to trunk crashes
my version of hbase-92 HRegionInfo.readFields() tries to find HTD in
HRegionInfo and fails)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4982) graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts
[ https://issues.apache.org/jira/browse/HBASE-4982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4982: - Attachment: 4982.txt Tried this. It works. graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts Key: HBASE-4982 URL: https://issues.apache.org/jira/browse/HBASE-4982 Project: HBase Issue Type: Bug Components: scripts Reporter: stack Attachments: 4982.txt Means, unless conf is in default location, we mess up asking zk for state (we'll be pointed at wrong ensemble), etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165458#comment-13165458 ] Lars Hofhansl commented on HBASE-4682: -- I did some more testing with this and everything works fine. Apparently there are no tests for import/export anywhere, so I am adding a test for existing functionality as well. Support deleted rows using Import/Export Key: HBASE-4682 URL: https://issues.apache.org/jira/browse/HBASE-4682 Project: HBase Issue Type: Sub-task Components: mapreduce Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4682-v1.txt Parent allows keeping deleted rows around. Would be nice if those could be exported and imported as well. All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4982) graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts
[ https://issues.apache.org/jira/browse/HBASE-4982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-4982. -- Resolution: Fixed Fix Version/s: 0.90.5 Assignee: stack Committed to TRUNK and the two branches. graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts Key: HBASE-4982 URL: https://issues.apache.org/jira/browse/HBASE-4982 Project: HBase Issue Type: Bug Components: scripts Reporter: stack Assignee: stack Fix For: 0.90.5 Attachments: 4982.txt Means, unless conf is in default location, we mess up asking zk for state (we'll be pointed at wrong ensemble), etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4969) tautology in HRegionInfo.readFields
[
https://issues.apache.org/jira/browse/HBASE-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165468#comment-13165468
]
Phabricator commented on HBASE-4969:
nspiegelberg has commented on the revision HBASE-4969 [jira] tautology in
HRegionInfo.readFields.
@stack: have you run 'mvn initialize -Darc' on that directory yet?
REVISION DETAIL
https://reviews.facebook.net/D669
tautology in HRegionInfo.readFields
---
Key: HBASE-4969
URL: https://issues.apache.org/jira/browse/HBASE-4969
Project: HBase
Issue Type: Bug
Reporter: Prakash Khemani
Assignee: Prakash Khemani
Attachments: HBASE-4969.D669.1.patch
In HRegionInfo.readFields() the following looks wrong to me
} else if (getVersion() == VERSION) {
it is always true.
Should it have been
} else if (getVersion() == version) {
version is a local variable where the deserialized-version is stored.
(I am struggling with another issue where after applying some patches -
including HBASE-4388 Second start after migration from 90 to trunk crashes
my version of hbase-92 HRegionInfo.readFields() tries to find HTD in
HRegionInfo and fails)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4969) tautology in HRegionInfo.readFields
[
https://issues.apache.org/jira/browse/HBASE-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165482#comment-13165482
]
Phabricator commented on HBASE-4969:
stack has commented on the revision HBASE-4969 [jira] tautology in
HRegionInfo.readFields.
@Nicolas
That made a big difference (I'll doc this in our book after I get it to
work). After installing cert which the tool guided me through nicely, I got to
this:
Exception:
ERR-CONDUIT-CORE: Bogus ID provided to load().
(Run with --trace for a full exception trace.)
h-24-30:hbase stack$ arc amend --trace --revision D669
Loading phutil library 'arclib' from
'/Users/Stack/checkouts/hbase/.arc_jira_lib'...
[0] conduit conduit.connect()
[0] conduit 418,950 us
[1] exec $ (cd '/Users/Stack/checkouts/hbase'; git rev-parse
--show-cdup)
[1] exec 10,368 us
[2] exec $ (cd '/Users/Stack/checkouts/hbase/'; git rev-parse --verify
HEAD^)
[2] exec 15,268 us
[3] exec $ (cd '/Users/Stack/checkouts/hbase/'; git diff --no-ext-diff
--no-textconv --raw 'HEAD^' --)
[4] exec $ (cd '/Users/Stack/checkouts/hbase/'; git diff --no-ext-diff
--no-textconv --raw HEAD --)
[5] exec $ (cd '/Users/Stack/checkouts/hbase/'; git ls-files --others
--exclude-standard)
[6] exec $ (cd '/Users/Stack/checkouts/hbase/'; git ls-files -m)
[6] exec 62,638 us
[3] exec 67,122 us
[5] exec 65,706 us
[4] exec 75,585 us
[7] conduit differential.getcommitmessage()
Fatal error: Uncaught exception 'ConduitClientException' with message
'ERR-CONDUIT-CORE: Bogus ID provided to load().' in
/Users/Stack/checkouts/libphutil/src/conduit/client/ConduitFuture.php:54
Stack trace:
#0 /Users/Stack/checkouts/libphutil/src/future/proxy/FutureProxy.php(62):
ConduitFuture-didReceiveResult(Array)
#1 /Users/Stack/checkouts/libphutil/src/future/proxy/FutureProxy.php(39):
FutureProxy-getResult()
#2 /Users/Stack/checkouts/libphutil/src/conduit/client/ConduitClient.php(52):
FutureProxy-resolve()
#3
/Users/Stack/checkouts/arcanist/src/workflow/amend/ArcanistAmendWorkflow.php(125):
ConduitClient-callMethodSynchronous('differential.ge...', Array)
#4 /Users/Stack/checkouts/arcanist/scripts/arcanist.php(257):
ArcanistAmendWorkflow-run()
#5 {main}
thrown in
/Users/Stack/checkouts/libphutil/src/conduit/client/ConduitFuture.php on line 54
REVISION DETAIL
https://reviews.facebook.net/D669
tautology in HRegionInfo.readFields
---
Key: HBASE-4969
URL: https://issues.apache.org/jira/browse/HBASE-4969
Project: HBase
Issue Type: Bug
Reporter: Prakash Khemani
Assignee: Prakash Khemani
Attachments: HBASE-4969.D669.1.patch
In HRegionInfo.readFields() the following looks wrong to me
} else if (getVersion() == VERSION) {
it is always true.
Should it have been
} else if (getVersion() == version) {
version is a local variable where the deserialized-version is stored.
(I am struggling with another issue where after applying some patches -
including HBASE-4388 Second start after migration from 90 to trunk crashes
my version of hbase-92 HRegionInfo.readFields() tries to find HTD in
HRegionInfo and fails)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4981) add raw scan support to shell
[
https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165526#comment-13165526
]
stack commented on HBASE-4981:
--
This is sweet. I like passing attribute at same time. Then could have RAW =
true and DEBUG = true. Issue I was talking of was HBASE-4981, a debug filter.
I was thinking it would something like the following (assuming both RAW and
DEBUG):
{code}
hbase(main):005:0 scan 'x1', {RAW=true, VERSIONS=10}
ROW COLUMN+CELL
r2 column=f:, timestamp=1323323616226, type=DeleteFamily
r2 column=f:c, timestamp=1323323617759, value=v3
r2 column=f:c, timestamp=1323323616226, value=v2
r2 column=f:c, timestamp=1323323614496, value=v1
r2 column=f:.values_skipped_returning_this_row,
timestamp=1323323614496, value=20021
r2 column=f:.storefiles_in_this_family,
timestamp=1323323614496, value=4
...
2 row(s) in 0.0500 seconds
{code}
i.e. we'd tag on some values on end of each row result that described things
like number of storefiles in the cf, and how many values we skipped returning a
row result (for example, if lots of deleted values skipped, would be nice to
show that somehow)
add raw scan support to shell
-
Key: HBASE-4981
URL: https://issues.apache.org/jira/browse/HBASE-4981
Project: HBase
Issue Type: Sub-task
Components: shell
Affects Versions: 0.94.0
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.94.0
Attachments: 4981-v1.txt
Parent adds raw scan support to include delete markers and deleted rows in
scan results. Would be nice it that would available in the shell to see
exactly what exists in a table.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165540#comment-13165540 ] Lars Hofhansl commented on HBASE-4981: -- Nice! The debug option would be extremely helpful. So I'll add the RAW stuff to the scan's help blurb, and then you'll continue in HBASE-4981? Do I get a +1 here? :) add raw scan support to shell - Key: HBASE-4981 URL: https://issues.apache.org/jira/browse/HBASE-4981 Project: HBase Issue Type: Sub-task Components: shell Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4981-v1.txt Parent adds raw scan support to include delete markers and deleted rows in scan results. Would be nice it that would available in the shell to see exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4984) index.html - adding a link to architecture overview in more information section.
[ https://issues.apache.org/jira/browse/HBASE-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165548#comment-13165548 ] Hudson commented on HBASE-4984: --- Integrated in HBase-TRUNK #2530 (See [https://builds.apache.org/job/HBase-TRUNK/2530/]) hbase-4984 index.html. adding link to arch overview in More Info section. index.html - adding a link to architecture overview in more information section. -- Key: HBASE-4984 URL: https://issues.apache.org/jira/browse/HBASE-4984 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: index_hbase_4984.xml.patch index.html * adding link to architecture overview in the book from index.html in the More Information? section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165565#comment-13165565 ] Lars Hofhansl commented on HBASE-4970: -- V2 looks good to me. +1 Is this good enough for you gaojinchao? It will put more burden on the application to manage the ExecutorService and HConnection. Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch) --- Key: HBASE-4970 URL: https://issues.apache.org/jira/browse/HBASE-4970 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.90.4 Reporter: gaojinchao Assignee: gaojinchao Priority: Trivial Fix For: 0.90.5 Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down. Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]? You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released. So I think we can add a parameter to change keepAliveTime of Htable thread pool. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4985) pom.xml - changing docbook section depth to 2 for chapters
pom.xml - changing docbook section depth to 2 for chapters -- Key: HBASE-4985 URL: https://issues.apache.org/jira/browse/HBASE-4985 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor pom.xml * changing to tocMaxDepth to 2 for chapters (will show 2 levels in Chapters, but still only 1 level in the main TOC). - tocSectionDepth1/tocSectionDepth + tocMaxDepth2/tocMaxDepth -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4682: - Attachment: 4682-v2.txt Addressed Ted's comment. Added an Import/Export test (there were NO tests for Import/Export before) Support deleted rows using Import/Export Key: HBASE-4682 URL: https://issues.apache.org/jira/browse/HBASE-4682 Project: HBase Issue Type: Sub-task Components: mapreduce Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4682-v1.txt, 4682-v2.txt Parent allows keeping deleted rows around. Would be nice if those could be exported and imported as well. All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4985) pom.xml - changing docbook section depth to 2 for chapters
[ https://issues.apache.org/jira/browse/HBASE-4985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165583#comment-13165583 ] stack commented on HBASE-4985: -- +1 pom.xml - changing docbook section depth to 2 for chapters -- Key: HBASE-4985 URL: https://issues.apache.org/jira/browse/HBASE-4985 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: pom_hbase_4985.xml.patch pom.xml * changing to tocMaxDepth to 2 for chapters (will show 2 levels in Chapters, but still only 1 level in the main TOC). - tocSectionDepth1/tocSectionDepth + tocMaxDepth2/tocMaxDepth -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4682: - Status: Patch Available (was: Open) Support deleted rows using Import/Export Key: HBASE-4682 URL: https://issues.apache.org/jira/browse/HBASE-4682 Project: HBase Issue Type: Sub-task Components: mapreduce Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4682-v1.txt, 4682-v2.txt Parent allows keeping deleted rows around. Would be nice if those could be exported and imported as well. All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4985) pom.xml - changing docbook section depth to 2 for chapters
[ https://issues.apache.org/jira/browse/HBASE-4985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4985: - Status: Patch Available (was: Open) pom.xml - changing docbook section depth to 2 for chapters -- Key: HBASE-4985 URL: https://issues.apache.org/jira/browse/HBASE-4985 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: pom_hbase_4985.xml.patch pom.xml * changing to tocMaxDepth to 2 for chapters (will show 2 levels in Chapters, but still only 1 level in the main TOC). - tocSectionDepth1/tocSectionDepth + tocMaxDepth2/tocMaxDepth -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4985) pom.xml - changing docbook section depth to 2 for chapters
[ https://issues.apache.org/jira/browse/HBASE-4985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4985: - Attachment: pom_hbase_4985.xml.patch pom.xml - changing docbook section depth to 2 for chapters -- Key: HBASE-4985 URL: https://issues.apache.org/jira/browse/HBASE-4985 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: pom_hbase_4985.xml.patch pom.xml * changing to tocMaxDepth to 2 for chapters (will show 2 levels in Chapters, but still only 1 level in the main TOC). - tocSectionDepth1/tocSectionDepth + tocMaxDepth2/tocMaxDepth -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4985) pom.xml - changing docbook section depth to 2 for chapters
[ https://issues.apache.org/jira/browse/HBASE-4985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4985: - Resolution: Fixed Status: Resolved (was: Patch Available) pom.xml - changing docbook section depth to 2 for chapters -- Key: HBASE-4985 URL: https://issues.apache.org/jira/browse/HBASE-4985 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: pom_hbase_4985.xml.patch pom.xml * changing to tocMaxDepth to 2 for chapters (will show 2 levels in Chapters, but still only 1 level in the main TOC). - tocSectionDepth1/tocSectionDepth + tocMaxDepth2/tocMaxDepth -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4682) Support deleted rows using Import/Export
[
https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165589#comment-13165589
]
stack commented on HBASE-4682:
--
Nice test. For sure its medium sized? Takes 50 seconds?
This bit seems a little messy:
{code}
-put.add(kv);
+if (kv.isDelete()) {
+ context.write(key, new Delete(kv));
+} else {
+ if (put == null) {
+put = new Put(key.get());
+ }
+ put.add(kv);
+}
}
- return put;
+ if (put != null) {
+context.write(key, put);
+ }
{code}
We write the context if its a delete but if a put, we make a put instance and
then do the context.write later... can they not both do context.write in same
place or share the context.write invocation (if context.write takes a Mutation).
Support deleted rows using Import/Export
Key: HBASE-4682
URL: https://issues.apache.org/jira/browse/HBASE-4682
Project: HBase
Issue Type: Sub-task
Components: mapreduce
Affects Versions: 0.94.0
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.94.0
Attachments: 4682-v1.txt, 4682-v2.txt
Parent allows keeping deleted rows around. Would be nice if those could be
exported and imported as well.
All the building blocks are there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4981: - Attachment: 4981-v2.txt Patch including help for scan command. add raw scan support to shell - Key: HBASE-4981 URL: https://issues.apache.org/jira/browse/HBASE-4981 Project: HBase Issue Type: Sub-task Components: shell Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4981-v1.txt, 4981-v2.txt Parent adds raw scan support to include delete markers and deleted rows in scan results. Would be nice it that would available in the shell to see exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165590#comment-13165590 ] stack commented on HBASE-4981: -- +1 add raw scan support to shell - Key: HBASE-4981 URL: https://issues.apache.org/jira/browse/HBASE-4981 Project: HBase Issue Type: Sub-task Components: shell Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4981-v1.txt, 4981-v2.txt Parent adds raw scan support to include delete markers and deleted rows in scan results. Would be nice it that would available in the shell to see exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165591#comment-13165591 ] stack commented on HBASE-4981: -- Oh, do you need to mention new facility in the Scan help for shell? If you can get it in, add on commit. add raw scan support to shell - Key: HBASE-4981 URL: https://issues.apache.org/jira/browse/HBASE-4981 Project: HBase Issue Type: Sub-task Components: shell Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4981-v1.txt, 4981-v2.txt Parent adds raw scan support to include delete markers and deleted rows in scan results. Would be nice it that would available in the shell to see exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4986) index.html - found another grammatical error
index.html - found another grammatical error Key: HBASE-4986 URL: https://issues.apache.org/jira/browse/HBASE-4986 Project: HBase Issue Type: Bug Reporter: Doug Meil Assignee: Doug Meil Priority: Trivial modeled after Google' should be modeled after Google's - it was missing the s -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4986) index.html - found another grammatical error
[ https://issues.apache.org/jira/browse/HBASE-4986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4986: - Attachment: index_4986.xml.patch index.html - found another grammatical error Key: HBASE-4986 URL: https://issues.apache.org/jira/browse/HBASE-4986 Project: HBase Issue Type: Bug Reporter: Doug Meil Assignee: Doug Meil Priority: Trivial Attachments: index_4986.xml.patch modeled after Google' should be modeled after Google's - it was missing the s -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4986) index.html - found another grammatical error
[ https://issues.apache.org/jira/browse/HBASE-4986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4986: - Status: Patch Available (was: Open) index.html - found another grammatical error Key: HBASE-4986 URL: https://issues.apache.org/jira/browse/HBASE-4986 Project: HBase Issue Type: Bug Reporter: Doug Meil Assignee: Doug Meil Priority: Trivial Attachments: index_4986.xml.patch modeled after Google' should be modeled after Google's - it was missing the s -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4986) index.html - found another grammatical error
[ https://issues.apache.org/jira/browse/HBASE-4986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4986: - Resolution: Fixed Status: Resolved (was: Patch Available) index.html - found another grammatical error Key: HBASE-4986 URL: https://issues.apache.org/jira/browse/HBASE-4986 Project: HBase Issue Type: Bug Reporter: Doug Meil Assignee: Doug Meil Priority: Trivial Attachments: index_4986.xml.patch modeled after Google' should be modeled after Google's - it was missing the s -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165606#comment-13165606 ] Lars Hofhansl commented on HBASE-4682: -- Heh... It takes 46 secs on my machine. Can make it large, too. As for the messy-ness... The problem is that Delete has strange rules as what you can do in a single Delete (when you add a deleteFamily all prior deleteColumn(s) are removed, etc). So deletes have to be written one-by-one to be correct (that is also why I added the Delete(kv) constructor, rather than an delete(kv) method). A put, on the other hand, allows adding all put-related KVs of the same row into a single Put object. That's why delete-kvs are issued immediately, and put-kvs are collected into a single Put (that's also what the existing code does). It is possible that particular row only has delete markers, so the Put is created when needed, and only written to the HBase when there were any put-kvs. Support deleted rows using Import/Export Key: HBASE-4682 URL: https://issues.apache.org/jira/browse/HBASE-4682 Project: HBase Issue Type: Sub-task Components: mapreduce Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4682-v1.txt, 4682-v2.txt Parent allows keeping deleted rows around. Would be nice if those could be exported and imported as well. All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165621#comment-13165621 ] stack commented on HBASE-4682: -- Thanks for explaination. +1 on commit. Support deleted rows using Import/Export Key: HBASE-4682 URL: https://issues.apache.org/jira/browse/HBASE-4682 Project: HBase Issue Type: Sub-task Components: mapreduce Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4682-v1.txt, 4682-v2.txt Parent allows keeping deleted rows around. Would be nice if those could be exported and imported as well. All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4973) On failure, HBaseAdmin sleeps one time too many
[
https://issues.apache.org/jira/browse/HBASE-4973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4973:
-
Resolution: Fixed
Fix Version/s: 0.94.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed TRUNK. Thanks for patch N.
On failure, HBaseAdmin sleeps one time too many
---
Key: HBASE-4973
URL: https://issues.apache.org/jira/browse/HBASE-4973
Project: HBase
Issue Type: Improvement
Components: client
Affects Versions: 0.94.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
Fix For: 0.94.0
Attachments: 4973_HBaseAdmin.patch
In this code last sleep is useless as we're not retrying. This can slow down
failure scenarios by a few seconds (up to 32 second).
{noformat}
public HBaseAdmin(Configuration c)
throws MasterNotRunningException, ZooKeeperConnectionException {
this.conf = HBaseConfiguration.create(c);
this.connection = HConnectionManager.getConnection(this.conf);
this.pause = this.conf.getLong(hbase.client.pause, 1000);
this.numRetries = this.conf.getInt(hbase.client.retries.number, 10);
this.retryLongerMultiplier = this.conf.getInt(
hbase.client.retries.longer.multiplier, 10);
int tries = 0;
for (; tries numRetries; ++tries) {
try {
this.connection.getMaster();
break;
} catch (MasterNotRunningException mnre) {
HConnectionManager.deleteStaleConnection(this.connection);
this.connection = HConnectionManager.getConnection(this.conf);
} catch (UndeclaredThrowableException ute) {
HConnectionManager.deleteStaleConnection(this.connection);
this.connection = HConnectionManager.getConnection(this.conf);
}
try { // Sleep
Thread.sleep(getPauseTime(tries));
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
// we should delete connection between client and zookeeper
HConnectionManager.deleteStaleConnection(this.connection);
throw new MasterNotRunningException(Interrupted);
}
}
if (tries = numRetries) {
// we should delete connection between client and zookeeper
HConnectionManager.deleteStaleConnection(this.connection);
throw new MasterNotRunningException(Retried + numRetries + times);
}
}
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165631#comment-13165631 ] Hadoop QA commented on HBASE-4682: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12506658/4682-v2.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -160 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 74 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/476//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/476//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/476//console This message is automatically generated. Support deleted rows using Import/Export Key: HBASE-4682 URL: https://issues.apache.org/jira/browse/HBASE-4682 Project: HBase Issue Type: Sub-task Components: mapreduce Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4682-v1.txt, 4682-v2.txt Parent allows keeping deleted rows around. Would be nice if those could be exported and imported as well. All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4980) Null pointer exception in HBaseClient receiveResponse
[
https://issues.apache.org/jira/browse/HBASE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4980:
-
Resolution: Fixed
Fix Version/s: 0.92.0
Assignee: Shrijeet Paliwal
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed trunk and branch. Thanks for the patch Shrjeet.
Null pointer exception in HBaseClient receiveResponse
-
Key: HBASE-4980
URL: https://issues.apache.org/jira/browse/HBASE-4980
Project: HBase
Issue Type: Bug
Components: client
Affects Versions: 0.92.0
Reporter: Shrijeet Paliwal
Assignee: Shrijeet Paliwal
Labels: newbie
Fix For: 0.92.0
Attachments:
0001-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
0002-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
Relevant Stack trace:
2011-11-30 13:10:26,557 [IPC Client (47) connection to
xx.xx.xx/172.22.4.68:60020 from an unknown user] WARN
org.apache.hadoop.ipc.HBaseClient - Unexpected exception receiving call
responses
java.lang.NullPointerException
-at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:583)
-at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:511)
{code}
if (LOG.isDebugEnabled())
LOG.debug(getName() + got value # + id);
Call call = calls.remove(id);
// Read the flag byte
byte flag = in.readByte();
boolean isError = ResponseFlag.isError(flag);
if (ResponseFlag.isLength(flag)) {
// Currently length if present is unused.
in.readInt();
}
int state = in.readInt(); // Read the state. Currently unused.
if (isError) {
//noinspection ThrowableInstanceNeverThrown
call.setException(new RemoteException( WritableUtils.readString(in),
WritableUtils.readString(in)));
} else {
{code}
This line {code}Call call = calls.remove(id);{code} may return a null
'call'. It is so because if you have rpc timeout enable, we proactively clean
up other calls which have expired their lifetime along with the call for
which socket timeout exception happend.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4965) Monitor the open file descriptors and the threads counters during the unit tests
[
https://issues.apache.org/jira/browse/HBASE-4965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165639#comment-13165639
]
stack commented on HBASE-4965:
--
On hadoop-qa being set to 1024 fds only, thats weird. We dump the ulimit
before the test starts and it shows:
{code}
Linux asf011.sp2.ygridcore.net 2.6.32-33-server #71-Ubuntu SMP Wed Jul 20
17:42:25 UTC 2011 x86_64 GNU/Linux
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 20
file size (blocks, -f) unlimited
pending signals (-i) 16382
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 6
pipe size(512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 2048
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
6
Running in Jenkins mode
{code}
... so 60k.
So, I wonder where disconnect between your finding and ulimit is? We're
running as a different user after ulimit is output?
I love that leaks report. Thats excellent.
Trying the patch locally
Monitor the open file descriptors and the threads counters during the unit
tests
Key: HBASE-4965
URL: https://issues.apache.org/jira/browse/HBASE-4965
Project: HBase
Issue Type: Improvement
Components: test
Affects Versions: 0.94.0
Environment: all
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
Attachments: 4965_all.patch, ResourceChecker.java,
ResourceCheckerJUnitRule.java
We're seeing a lot of issues with hadoop-qa related to threads or file
descriptors.
Monitoring these counters would ease the analysis.
Note as well that
- if we want to execute the tests in the same jvm (because the test is small
or because we want to share the cluster) we can't afford to leak too many
resources
- if the tests leak, it's more difficult to detect a leak in the software
itself.
I attach piece of code that I used. It requires two lines in a unit test
class to:
- before every test, count the threads and the open file descriptor
- after every test, compare with the previous value.
I ran it on some tests; we have for example:
- client.TestMultiParallel#testBatchWithManyColsInOneRowGetAndPut: 232
threads (was 231), 390 file descriptors (was 390). = TestMultiParallel uses
232 threads!
- client.TestMultipleTimestamps#testWithColumnDeletes: 152 threads (was 151),
283 file descriptors (was 282).
- client.TestAdmin#testCheckHBaseAvailableClosesConnection: 477 threads (was
294), 815 file descriptors (was 461)
- client.TestMetaMigrationRemovingHTD#testMetaMigration: 149 threads (was
148), 310 file descriptors (was 307).
It's not always leaks, we can expect some pooling effects. But still...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4981: - Attachment: 4981-v3.txt Final change that was committed. add raw scan support to shell - Key: HBASE-4981 URL: https://issues.apache.org/jira/browse/HBASE-4981 Project: HBase Issue Type: Sub-task Components: shell Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4981-v1.txt, 4981-v2.txt, 4981-v3.txt Parent adds raw scan support to include delete markers and deleted rows in scan results. Would be nice it that would available in the shell to see exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-4981. -- Resolution: Fixed Hadoop Flags: Reviewed Thanks for the review stack. add raw scan support to shell - Key: HBASE-4981 URL: https://issues.apache.org/jira/browse/HBASE-4981 Project: HBase Issue Type: Sub-task Components: shell Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4981-v1.txt, 4981-v2.txt, 4981-v3.txt Parent adds raw scan support to include delete markers and deleted rows in scan results. Would be nice it that would available in the shell to see exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4682: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk Support deleted rows using Import/Export Key: HBASE-4682 URL: https://issues.apache.org/jira/browse/HBASE-4682 Project: HBase Issue Type: Sub-task Components: mapreduce Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4682-v1.txt, 4682-v2.txt Parent allows keeping deleted rows around. Would be nice if those could be exported and imported as well. All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165679#comment-13165679 ] Lars Hofhansl commented on HBASE-4682: -- Thanks for the review stack (I added comments to the pieces in Import) Support deleted rows using Import/Export Key: HBASE-4682 URL: https://issues.apache.org/jira/browse/HBASE-4682 Project: HBase Issue Type: Sub-task Components: mapreduce Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4682-v1.txt, 4682-v2.txt Parent allows keeping deleted rows around. Would be nice if those could be exported and imported as well. All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165694#comment-13165694 ] Lars Hofhansl commented on HBASE-4981: -- bq. This is sweet. I like passing attribute at same time. Then could have RAW = true and DEBUG = true. Issue I was talking of was HBASE-4981, a debug filter. Actually HBASE-4981 is this one :)... You mean HBASE-4926? add raw scan support to shell - Key: HBASE-4981 URL: https://issues.apache.org/jira/browse/HBASE-4981 Project: HBase Issue Type: Sub-task Components: shell Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4981-v1.txt, 4981-v2.txt, 4981-v3.txt Parent adds raw scan support to include delete markers and deleted rows in scan results. Would be nice it that would available in the shell to see exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165695#comment-13165695 ] stack commented on HBASE-4981: -- duh add raw scan support to shell - Key: HBASE-4981 URL: https://issues.apache.org/jira/browse/HBASE-4981 Project: HBase Issue Type: Sub-task Components: shell Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 4981-v1.txt, 4981-v2.txt, 4981-v3.txt Parent adds raw scan support to include delete markers and deleted rows in scan results. Would be nice it that would available in the shell to see exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4982) graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts
[ https://issues.apache.org/jira/browse/HBASE-4982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165706#comment-13165706 ] Hudson commented on HBASE-4982: --- Integrated in HBase-TRUNK #2531 (See [https://builds.apache.org/job/HBase-TRUNK/2531/]) HBASE-4982 graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts stack : Files : * /hbase/trunk/bin/graceful_stop.sh graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts Key: HBASE-4982 URL: https://issues.apache.org/jira/browse/HBASE-4982 Project: HBase Issue Type: Bug Components: scripts Reporter: stack Assignee: stack Fix For: 0.90.5 Attachments: 4982.txt Means, unless conf is in default location, we mess up asking zk for state (we'll be pointed at wrong ensemble), etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-4352) Apply version of hbase-4015 to branch
[ https://issues.apache.org/jira/browse/HBASE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-4352: -- Reopened after reverting patch from 0.90 branch. Apply version of hbase-4015 to branch - Key: HBASE-4352 URL: https://issues.apache.org/jira/browse/HBASE-4352 Project: HBase Issue Type: Bug Reporter: stack Assignee: ramkrishna.s.vasudevan Priority: Blocker Attachments: HBASE-4352_0.90.patch, HBASE-4352_0.90_1.patch Consider adding a version of hbase-4015 to 0.90. It changes HRegionInterface so would need move change to end of the Interface and then test that it doesn't break rolling restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4352) Apply version of hbase-4015 to branch
[ https://issues.apache.org/jira/browse/HBASE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4352: - Fix Version/s: (was: 0.90.5) Apply version of hbase-4015 to branch - Key: HBASE-4352 URL: https://issues.apache.org/jira/browse/HBASE-4352 Project: HBase Issue Type: Bug Reporter: stack Assignee: ramkrishna.s.vasudevan Priority: Blocker Attachments: HBASE-4352_0.90.patch, HBASE-4352_0.90_1.patch Consider adding a version of hbase-4015 to 0.90. It changes HRegionInterface so would need move change to end of the Interface and then test that it doesn't break rolling restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4352) Apply version of hbase-4015 to branch
[ https://issues.apache.org/jira/browse/HBASE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165724#comment-13165724 ] ramkrishna.s.vasudevan commented on HBASE-4352: --- @Stack Where is the NPE coming in the client side? Could you paste the stack trace to chk it out. Apply version of hbase-4015 to branch - Key: HBASE-4352 URL: https://issues.apache.org/jira/browse/HBASE-4352 Project: HBase Issue Type: Bug Reporter: stack Assignee: ramkrishna.s.vasudevan Priority: Blocker Attachments: HBASE-4352_0.90.patch, HBASE-4352_0.90_1.patch Consider adding a version of hbase-4015 to 0.90. It changes HRegionInterface so would need move change to end of the Interface and then test that it doesn't break rolling restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4880) Region is on service before openRegionHandler completes, may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chunhui shen updated HBASE-4880:
Attachment: hbase-4880v4.patch
Region is on service before openRegionHandler completes, may cause data loss
Key: HBASE-4880
URL: https://issues.apache.org/jira/browse/HBASE-4880
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0, 0.94.0
Reporter: chunhui shen
Assignee: chunhui shen
Attachments: 4880.txt, hbase-4880.patch, hbase-4880v2.patch,
hbase-4880v3.patch, hbase-4880v4.patch
OpenRegionHandler in regionserver is processed as the following steps:
{code}
1.openregion()(Through it, closed = false, closing = false)
2.addToOnlineRegions(region)
3.update .meta. table
4.update ZK's node state to RS_ZK_REGION_OPEND
{code}
We can find that region is on service before Step 4.
It means client could put data to this region after step 3.
What will happen if step 4 is failed processing?
It will execute OpenRegionHandler#cleanupFailedOpen which will do closing
region, and master assign this region to another regionserver.
If closing region is failed, the data which is put between step 3 and step 4
may loss, because the region has been opend on another regionserver and be
put new data. Therefore, it may not be recoverd through replayRecoveredEdit()
because the edit's LogSeqId is smaller than current region SeqId.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4880) Region is on service before openRegionHandler completes, may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165737#comment-13165737
]
chunhui shen commented on HBASE-4880:
-
Address Ram and Stack comments in patchv4
Thanks.
Please check
Region is on service before openRegionHandler completes, may cause data loss
Key: HBASE-4880
URL: https://issues.apache.org/jira/browse/HBASE-4880
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0, 0.94.0
Reporter: chunhui shen
Assignee: chunhui shen
Attachments: 4880.txt, hbase-4880.patch, hbase-4880v2.patch,
hbase-4880v3.patch, hbase-4880v4.patch
OpenRegionHandler in regionserver is processed as the following steps:
{code}
1.openregion()(Through it, closed = false, closing = false)
2.addToOnlineRegions(region)
3.update .meta. table
4.update ZK's node state to RS_ZK_REGION_OPEND
{code}
We can find that region is on service before Step 4.
It means client could put data to this region after step 3.
What will happen if step 4 is failed processing?
It will execute OpenRegionHandler#cleanupFailedOpen which will do closing
region, and master assign this region to another regionserver.
If closing region is failed, the data which is put between step 3 and step 4
may loss, because the region has been opend on another regionserver and be
put new data. Therefore, it may not be recoverd through replayRecoveredEdit()
because the edit's LogSeqId is smaller than current region SeqId.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4987) wrong use of incarnation var in SplitLogManager
wrong use of incarnation var in SplitLogManager --- Key: HBASE-4987 URL: https://issues.apache.org/jira/browse/HBASE-4987 Project: HBase Issue Type: Bug Reporter: Prakash Khemani Assignee: Prakash Khemani @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165766#comment-13165766 ] Prakash Khemani commented on HBASE-4987: Old issue HBASE-4855 wrong use of incarnation var in SplitLogManager --- Key: HBASE-4987 URL: https://issues.apache.org/jira/browse/HBASE-4987 Project: HBase Issue Type: Bug Reporter: Prakash Khemani Assignee: Prakash Khemani @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4982) graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts
[ https://issues.apache.org/jira/browse/HBASE-4982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165767#comment-13165767 ] Hudson commented on HBASE-4982: --- Integrated in HBase-0.92 #178 (See [https://builds.apache.org/job/HBase-0.92/178/]) HBASE-4982 graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/bin/graceful_stop.sh graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts Key: HBASE-4982 URL: https://issues.apache.org/jira/browse/HBASE-4982 Project: HBase Issue Type: Bug Components: scripts Reporter: stack Assignee: stack Fix For: 0.90.5 Attachments: 4982.txt Means, unless conf is in default location, we mess up asking zk for state (we'll be pointed at wrong ensemble), etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4880) Region is on service before openRegionHandler completes, may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165779#comment-13165779
]
Hadoop QA commented on HBASE-4880:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12506692/hbase-4880v4.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -160 warning
messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 74 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/477//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/477//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/477//console
This message is automatically generated.
Region is on service before openRegionHandler completes, may cause data loss
Key: HBASE-4880
URL: https://issues.apache.org/jira/browse/HBASE-4880
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0, 0.94.0
Reporter: chunhui shen
Assignee: chunhui shen
Attachments: 4880.txt, hbase-4880.patch, hbase-4880v2.patch,
hbase-4880v3.patch, hbase-4880v4.patch
OpenRegionHandler in regionserver is processed as the following steps:
{code}
1.openregion()(Through it, closed = false, closing = false)
2.addToOnlineRegions(region)
3.update .meta. table
4.update ZK's node state to RS_ZK_REGION_OPEND
{code}
We can find that region is on service before Step 4.
It means client could put data to this region after step 3.
What will happen if step 4 is failed processing?
It will execute OpenRegionHandler#cleanupFailedOpen which will do closing
region, and master assign this region to another regionserver.
If closing region is failed, the data which is put between step 3 and step 4
may loss, because the region has been opend on another regionserver and be
put new data. Therefore, it may not be recoverd through replayRecoveredEdit()
because the edit's LogSeqId is smaller than current region SeqId.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4880) Region is on service before openRegionHandler completes, may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165783#comment-13165783
]
ramkrishna.s.vasudevan commented on HBASE-4880:
---
+1 on patch.
Region is on service before openRegionHandler completes, may cause data loss
Key: HBASE-4880
URL: https://issues.apache.org/jira/browse/HBASE-4880
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0, 0.94.0
Reporter: chunhui shen
Assignee: chunhui shen
Attachments: 4880.txt, hbase-4880.patch, hbase-4880v2.patch,
hbase-4880v3.patch, hbase-4880v4.patch
OpenRegionHandler in regionserver is processed as the following steps:
{code}
1.openregion()(Through it, closed = false, closing = false)
2.addToOnlineRegions(region)
3.update .meta. table
4.update ZK's node state to RS_ZK_REGION_OPEND
{code}
We can find that region is on service before Step 4.
It means client could put data to this region after step 3.
What will happen if step 4 is failed processing?
It will execute OpenRegionHandler#cleanupFailedOpen which will do closing
region, and master assign this region to another regionserver.
If closing region is failed, the data which is put between step 3 and step 4
may loss, because the region has been opend on another regionserver and be
put new data. Therefore, it may not be recoverd through replayRecoveredEdit()
because the edit's LogSeqId is smaller than current region SeqId.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-4987: --- Attachment: HBASE-4987.D675.1.patch khemani requested code review of HBASE-4987 [jira] wrong use of incarnation var in SplitLogManager. Reviewers: tedyu, nspiegelberg, dhruba, JIRA redo HBASE-4855 TEST PLAN TestSplitLogManager REVISION DETAIL https://reviews.facebook.net/D675 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/1485/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. wrong use of incarnation var in SplitLogManager --- Key: HBASE-4987 URL: https://issues.apache.org/jira/browse/HBASE-4987 Project: HBase Issue Type: Bug Reporter: Prakash Khemani Assignee: Prakash Khemani Attachments: HBASE-4987.D675.1.patch @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165802#comment-13165802 ] gaojinchao commented on HBASE-4970: --- I also want to add a parameter to change keepAliveTime of Htable thread pool. so that clients can have more option Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch) --- Key: HBASE-4970 URL: https://issues.apache.org/jira/browse/HBASE-4970 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.90.4 Reporter: gaojinchao Assignee: gaojinchao Priority: Trivial Fix For: 0.90.5 Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down. Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]? You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released. So I think we can add a parameter to change keepAliveTime of Htable thread pool. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4988) MetaServer crash cause all splitting regionserver abort
MetaServer crash cause all splitting regionserver abort
---
Key: HBASE-4988
URL: https://issues.apache.org/jira/browse/HBASE-4988
Project: HBase
Issue Type: Bug
Reporter: chunhui shen
If metaserver crash now,
All the splitting regionserver will abort theirself.
Becasue the code
{code}
this.journal.add(JournalEntry.PONR);
MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
this.parent.getRegionInfo(), a.getRegionInfo(), b.getRegionInfo());
{code}
If the JournalEntry is PONR, split's roll back will abort itselef.
It is terrible in huge putting environment when metaserver crash
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4988) MetaServer crash cause all splitting regionserver abort
[
https://issues.apache.org/jira/browse/HBASE-4988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165805#comment-13165805
]
chunhui shen commented on HBASE-4988:
-
logs
{code}
2011-12-07 17:49:17,737 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Onlined
writetest,28TPVACCO3EI47TH472E1997TX1ZDFQ7XUCMBA2LUKOD7G0U3NQ2L2FG0ILRGZ5ETHFESE5QIMFN8ONUDUXB80G7MEK58G7YM4EG,1323251351741.6399c204b8d45568a782fd0157d6700d.;
next sequenceid=3483318538
2011-12-07 17:49:17,737 DEBUG
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested
for
writetest,2FB\xC0EE\xC2LDFG\xC8\xB6GV\xCE\xC6F4\xBBE\xC87BM\xC0\xCD\xC3\xC8A\xB3\xCE\xD5G\xCBI\xBA\xBB\xCB\xD7R\xD2=\xC52U;P\xD2D\xCD\xBA\xC6\xC6A\xC1KI\xCDND\xC8\xCEKG\xC3\xCC\xCD\xB4\xC1=\xD0\xC4\xD2FSSPE\xD0V\xCE5@\xBCCN\xC4\xCB\xBE7L\xC8E\xC1\xBD\xCFH,1323251351741.a639e2eda8b2de9ca368c1a13ebbcb44.
because Region has references on open; priority=16, compaction queue size=1
2011-12-07 17:49:17,737 INFO org.apache.hadoop.hbase.catalog.CatalogTracker:
Failed verification of .META.,,1 at address=dw83.kgb.sqa.cm4:60020;
java.io.EOFException
2011-12-07 17:49:17,737 INFO org.apache.hadoop.hbase.catalog.CatalogTracker:
Current cached META location is not valid, resetting
2011-12-07 17:49:17,740 INFO org.apache.hadoop.hbase.catalog.CatalogTracker:
Failed verification of .META.,,1 at address=dw83.kgb.sqa.cm4:60020;
java.net.ConnectException: Connection refused
2011-12-07 17:49:17,740 WARN
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Running rollback of
failed split of
writetest,28TPVACCO3EI47TH472E1997TX1ZDFQ7XUCMBA2LUKOD7G0U3NQ2L2FG0ILRGZ5ETHFESE5QIMFN8ONUDUXB80G7MEK58G7YM4EG,1323240352298.c7bde4437e5b12bc7226485dcbc2700b.;
Timed out (2147483647ms)
2011-12-07 17:49:17,740 FATAL
org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
serverName=dw87.kgb.sqa.cm4,60020,1323244700069, load=(requests=393,
regions=12, usedHeap=742, maxHeap=15872): Abort; we got an error after
point-of-no-return
{code}
MetaServer crash cause all splitting regionserver abort
---
Key: HBASE-4988
URL: https://issues.apache.org/jira/browse/HBASE-4988
Project: HBase
Issue Type: Bug
Reporter: chunhui shen
If metaserver crash now,
All the splitting regionserver will abort theirself.
Becasue the code
{code}
this.journal.add(JournalEntry.PONR);
MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
this.parent.getRegionInfo(), a.getRegionInfo(),
b.getRegionInfo());
{code}
If the JournalEntry is PONR, split's roll back will abort itselef.
It is terrible in huge putting environment when metaserver crash
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4988) MetaServer crash cause all splitting regionserver abort
[
https://issues.apache.org/jira/browse/HBASE-4988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chunhui shen updated HBASE-4988:
Attachment: hbase-4988v1.patch
MetaServer crash cause all splitting regionserver abort
---
Key: HBASE-4988
URL: https://issues.apache.org/jira/browse/HBASE-4988
Project: HBase
Issue Type: Bug
Reporter: chunhui shen
Attachments: hbase-4988v1.patch
If metaserver crash now,
All the splitting regionserver will abort theirself.
Becasue the code
{code}
this.journal.add(JournalEntry.PONR);
MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
this.parent.getRegionInfo(), a.getRegionInfo(),
b.getRegionInfo());
{code}
If the JournalEntry is PONR, split's roll back will abort itselef.
It is terrible in huge putting environment when metaserver crash
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-4987: --- Attachment: HBASE-4987.D675.2.patch khemani updated the revision HBASE-4987 [jira] wrong use of incarnation var in SplitLogManager. Reviewers: tedyu, nspiegelberg, dhruba, JIRA synchronization on task is needed REVISION DETAIL https://reviews.facebook.net/D675 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java wrong use of incarnation var in SplitLogManager --- Key: HBASE-4987 URL: https://issues.apache.org/jira/browse/HBASE-4987 Project: HBase Issue Type: Bug Reporter: Prakash Khemani Assignee: Prakash Khemani Attachments: HBASE-4987.D675.1.patch, HBASE-4987.D675.2.patch @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4988) MetaServer crash cause all splitting regionserver abort
[
https://issues.apache.org/jira/browse/HBASE-4988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165811#comment-13165811
]
chunhui shen commented on HBASE-4988:
-
In patchv1,
when doing MetaEditor.offlineParentInMeta, if metaserver is not ok now ,
doesn't add JournalEntry.PONR.
Therefore, rollback won't abort itself
MetaServer crash cause all splitting regionserver abort
---
Key: HBASE-4988
URL: https://issues.apache.org/jira/browse/HBASE-4988
Project: HBase
Issue Type: Bug
Reporter: chunhui shen
Attachments: hbase-4988v1.patch
If metaserver crash now,
All the splitting regionserver will abort theirself.
Becasue the code
{code}
this.journal.add(JournalEntry.PONR);
MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
this.parent.getRegionInfo(), a.getRegionInfo(),
b.getRegionInfo());
{code}
If the JournalEntry is PONR, split's roll back will abort itselef.
It is terrible in huge putting environment when metaserver crash
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4352) Apply version of hbase-4015 to branch
[
https://issues.apache.org/jira/browse/HBASE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165828#comment-13165828
]
stack commented on HBASE-4352:
--
Here is a 0.90.5 master trying to reload a regionserver that has been restarted
as a 0.90.5 (it had been a 0.90.4):
{code}
2011-12-08 22:56:53,807 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Assigning region
TestTable,0013681953,1323325588029.5d0a564b65a53b1c86f7c02407c82231. to
sv4r9s38,7003,1323384867368
2011-12-08 22:56:53,809 WARN org.apache.hadoop.ipc.HBaseClient: Unexpected
exception receiving call responses
java.lang.NullPointerException
2011-12-08 22:56:53,810 WARN org.apache.hadoop.hbase.master.AssignmentManager:
Failed assignment of
TestTable,0013681953,1323325588029.5d0a564b65a53b1c86f7c02407c82231. to
serverName=sv4r9s38,7003,1323384867368, load=(requests=0, regions=0,
usedHeap=23, maxHeap=493), trying to assign elsewhere instead; retry=0
java.io.IOException: Call to /10.4.9.38:7003 failed on local exception:
java.io.IOException: Unexpected exception receiving call responses
at
org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:806)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:775)
at
org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
at $Proxy7.openRegion(Unknown Source)
at
org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:561)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1043)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:858)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:838)
at
org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:92)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:156)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Unexpected exception receiving call responses
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:481)
Caused by: java.lang.NullPointerException
{code}
It looks like call is null as per another issue that just got committed.
Shouldn't be.
Was getting bunches of this on regionserver side, though it don't seem directly
related:
{code}
2011-12-08 22:56:53,809 DEBUG
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Processing open
of TestTable,0013681953,1323325588029.5d0a564b65a53b1c86f7c02407c82231.
2011-12-08 22:56:53,810 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:7003-0x1341fa8d95400cc Attempting to transition node
5d0a564b65a53b1c86f7c02407c82231 from M_ZK_REGION_OFFLINE to
RS_ZK_REGION_OPENING
2011-12-08 22:56:53,811 WARN org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:7003-0x1341fa8d95400cc Attempt to transition the unassigned node
for 5d0a564b65a53b1c86f7c02407c82231 from M_ZK_REGION_OFFLINE to
RS_ZK_REGION_OPENING failed, the node existed but was version 2 not the
expected version -1
2011-12-08 22:56:53,811 WARN
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed
transition from OFFLINE to OPENING for region=5d0a564b65a53b1c86f7c02407c82231
2011-12-08 22:56:53,811 WARN
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Region was
hijacked? It no longer exists, encodedName=5d0a564b65a53b1c86f7c02407c82231
{code}
I was hoping to put up a 0.90.5RC. If you want me to hold a while, I can.
Else, I'll go ahead and cut an RC w/o this.
Apply version of hbase-4015 to branch
-
Key: HBASE-4352
URL: https://issues.apache.org/jira/browse/HBASE-4352
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
Attachments: HBASE-4352_0.90.patch, HBASE-4352_0.90_1.patch
Consider adding a version of hbase-4015 to 0.90. It changes HRegionInterface
so would need move change to end of the Interface and then test that it
doesn't break rolling restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4880) Region is on service before openRegionHandler completes, may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165837#comment-13165837
]
Ted Yu commented on HBASE-4880:
---
+1 on patch v4.
Region is on service before openRegionHandler completes, may cause data loss
Key: HBASE-4880
URL: https://issues.apache.org/jira/browse/HBASE-4880
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0, 0.94.0
Reporter: chunhui shen
Assignee: chunhui shen
Attachments: 4880.txt, hbase-4880.patch, hbase-4880v2.patch,
hbase-4880v3.patch, hbase-4880v4.patch
OpenRegionHandler in regionserver is processed as the following steps:
{code}
1.openregion()(Through it, closed = false, closing = false)
2.addToOnlineRegions(region)
3.update .meta. table
4.update ZK's node state to RS_ZK_REGION_OPEND
{code}
We can find that region is on service before Step 4.
It means client could put data to this region after step 3.
What will happen if step 4 is failed processing?
It will execute OpenRegionHandler#cleanupFailedOpen which will do closing
region, and master assign this region to another regionserver.
If closing region is failed, the data which is put between step 3 and step 4
may loss, because the region has been opend on another regionserver and be
put new data. Therefore, it may not be recoverd through replayRecoveredEdit()
because the edit's LogSeqId is smaller than current region SeqId.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to d
[
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165839#comment-13165839
]
Ted Yu commented on HBASE-4946:
---
Will commit patch v4 tomorrow.
HTable.coprocessorExec (and possibly coprocessorProxy) does not work with
dynamically loaded coprocessors (from hdfs or local system), because the RPC
system tries to deserialize an unknown class.
-
Key: HBASE-4946
URL: https://issues.apache.org/jira/browse/HBASE-4946
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.0
Reporter: Andrei Dragomir
Assignee: Andrei Dragomir
Attachments: 4946-v4.txt, HBASE-4946-v2.patch, HBASE-4946-v3.patch,
HBASE-4946.patch
Loading coprocessors jars from hdfs works fine. I load it from the shell,
after setting the attribute, and it gets loaded:
{noformat}
INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor
config now ...
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class
com.MyCoprocessorClass needs to be loaded from a file -
hdfs://localhost:9000/coproc/rt- 0.0.1-SNAPSHOT.jar.
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance:
com.MyCoprocessorClass
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost:
RegionEnvironment createEnvironment
DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol
handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f.
protocol=com.MyCoprocessorClassProtocol
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load
coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
{noformat}
The problem is that this coprocessors simply extends BaseEndpointCoprocessor,
with a dynamic method. When calling this method from the client with
HTable.coprocessorExec, I get errors on the HRegionServer, because the call
cannot be deserialized from writables.
The problem is that Exec tries to do an early resolve of the coprocessor
class. The coprocessor class is loaded, but it is in the context of the
HRegionServer / HRegion. So, the call fails:
{noformat}
2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable:
Error in readFields
java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
at
org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
... 10 more
{noformat}
Probably the correct way to fix this is to make Exec really smart, so that it
knows all the class definitions loaded in CoprocessorHost(s).
I created a small patch that simply doesn't resolve the class definition in
the Exec, instead passing it as string down to the HRegion layer. This layer
knows all the definitions, and simply loads it by name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see:
[jira] [Updated] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4987: -- Status: Patch Available (was: Open) wrong use of incarnation var in SplitLogManager --- Key: HBASE-4987 URL: https://issues.apache.org/jira/browse/HBASE-4987 Project: HBase Issue Type: Bug Reporter: Prakash Khemani Assignee: Prakash Khemani Attachments: HBASE-4987.D675.1.patch, HBASE-4987.D675.2.patch @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165842#comment-13165842 ] Phabricator commented on HBASE-4987: tedyu has commented on the revision HBASE-4987 [jira] wrong use of incarnation var in SplitLogManager. The new patch relies on oldtask.deleted to not double increment batch.installed Interesting. REVISION DETAIL https://reviews.facebook.net/D675 wrong use of incarnation var in SplitLogManager --- Key: HBASE-4987 URL: https://issues.apache.org/jira/browse/HBASE-4987 Project: HBase Issue Type: Bug Reporter: Prakash Khemani Assignee: Prakash Khemani Attachments: HBASE-4987.D675.1.patch, HBASE-4987.D675.2.patch @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to d
[
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165841#comment-13165841
]
Lars Hofhansl commented on HBASE-4946:
--
Please don't comment out code, but remove it instead. We have a record of the
old code still in source control.
HTable.coprocessorExec (and possibly coprocessorProxy) does not work with
dynamically loaded coprocessors (from hdfs or local system), because the RPC
system tries to deserialize an unknown class.
-
Key: HBASE-4946
URL: https://issues.apache.org/jira/browse/HBASE-4946
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.0
Reporter: Andrei Dragomir
Assignee: Andrei Dragomir
Attachments: 4946-v4.txt, HBASE-4946-v2.patch, HBASE-4946-v3.patch,
HBASE-4946.patch
Loading coprocessors jars from hdfs works fine. I load it from the shell,
after setting the attribute, and it gets loaded:
{noformat}
INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor
config now ...
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class
com.MyCoprocessorClass needs to be loaded from a file -
hdfs://localhost:9000/coproc/rt- 0.0.1-SNAPSHOT.jar.
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance:
com.MyCoprocessorClass
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost:
RegionEnvironment createEnvironment
DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol
handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f.
protocol=com.MyCoprocessorClassProtocol
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load
coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
{noformat}
The problem is that this coprocessors simply extends BaseEndpointCoprocessor,
with a dynamic method. When calling this method from the client with
HTable.coprocessorExec, I get errors on the HRegionServer, because the call
cannot be deserialized from writables.
The problem is that Exec tries to do an early resolve of the coprocessor
class. The coprocessor class is loaded, but it is in the context of the
HRegionServer / HRegion. So, the call fails:
{noformat}
2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable:
Error in readFields
java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
at
org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
... 10 more
{noformat}
Probably the correct way to fix this is to make Exec really smart, so that it
knows all the class definitions loaded in CoprocessorHost(s).
I created a small patch that simply doesn't resolve the class definition in
the Exec, instead passing it as string down to the HRegion layer. This layer
knows all the definitions, and simply loads it by name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
[jira] [Updated] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to des
[
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-4946:
--
Attachment: 4946-v5.txt
Patch v5 removes commented out code.
HTable.coprocessorExec (and possibly coprocessorProxy) does not work with
dynamically loaded coprocessors (from hdfs or local system), because the RPC
system tries to deserialize an unknown class.
-
Key: HBASE-4946
URL: https://issues.apache.org/jira/browse/HBASE-4946
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.0
Reporter: Andrei Dragomir
Assignee: Andrei Dragomir
Attachments: 4946-v4.txt, 4946-v5.txt, HBASE-4946-v2.patch,
HBASE-4946-v3.patch, HBASE-4946.patch
Loading coprocessors jars from hdfs works fine. I load it from the shell,
after setting the attribute, and it gets loaded:
{noformat}
INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor
config now ...
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class
com.MyCoprocessorClass needs to be loaded from a file -
hdfs://localhost:9000/coproc/rt- 0.0.1-SNAPSHOT.jar.
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance:
com.MyCoprocessorClass
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost:
RegionEnvironment createEnvironment
DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol
handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f.
protocol=com.MyCoprocessorClassProtocol
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load
coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
{noformat}
The problem is that this coprocessors simply extends BaseEndpointCoprocessor,
with a dynamic method. When calling this method from the client with
HTable.coprocessorExec, I get errors on the HRegionServer, because the call
cannot be deserialized from writables.
The problem is that Exec tries to do an early resolve of the coprocessor
class. The coprocessor class is loaded, but it is in the context of the
HRegionServer / HRegion. So, the call fails:
{noformat}
2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable:
Error in readFields
java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
at
org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
... 10 more
{noformat}
Probably the correct way to fix this is to make Exec really smart, so that it
knows all the class definitions loaded in CoprocessorHost(s).
I created a small patch that simply doesn't resolve the class definition in
the Exec, instead passing it as string down to the HRegion layer. This layer
knows all the definitions, and simply loads it by name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see:
[jira] [Commented] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to d
[
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165851#comment-13165851
]
Lars Hofhansl commented on HBASE-4946:
--
+1 v5 lgtm
HTable.coprocessorExec (and possibly coprocessorProxy) does not work with
dynamically loaded coprocessors (from hdfs or local system), because the RPC
system tries to deserialize an unknown class.
-
Key: HBASE-4946
URL: https://issues.apache.org/jira/browse/HBASE-4946
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.0
Reporter: Andrei Dragomir
Assignee: Andrei Dragomir
Attachments: 4946-v4.txt, 4946-v5.txt, HBASE-4946-v2.patch,
HBASE-4946-v3.patch, HBASE-4946.patch
Loading coprocessors jars from hdfs works fine. I load it from the shell,
after setting the attribute, and it gets loaded:
{noformat}
INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor
config now ...
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class
com.MyCoprocessorClass needs to be loaded from a file -
hdfs://localhost:9000/coproc/rt- 0.0.1-SNAPSHOT.jar.
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance:
com.MyCoprocessorClass
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost:
RegionEnvironment createEnvironment
DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol
handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f.
protocol=com.MyCoprocessorClassProtocol
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load
coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
{noformat}
The problem is that this coprocessors simply extends BaseEndpointCoprocessor,
with a dynamic method. When calling this method from the client with
HTable.coprocessorExec, I get errors on the HRegionServer, because the call
cannot be deserialized from writables.
The problem is that Exec tries to do an early resolve of the coprocessor
class. The coprocessor class is loaded, but it is in the context of the
HRegionServer / HRegion. So, the call fails:
{noformat}
2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable:
Error in readFields
java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
at
org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
... 10 more
{noformat}
Probably the correct way to fix this is to make Exec really smart, so that it
knows all the class definitions loaded in CoprocessorHost(s).
I created a small patch that simply doesn't resolve the class definition in
the Exec, instead passing it as string down to the HRegion layer. This layer
knows all the definitions, and simply loads it by name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see:
[jira] [Commented] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165853#comment-13165853 ] Phabricator commented on HBASE-4987: stack has commented on the revision HBASE-4987 [jira] wrong use of incarnation var in SplitLogManager. +1 REVISION DETAIL https://reviews.facebook.net/D675 wrong use of incarnation var in SplitLogManager --- Key: HBASE-4987 URL: https://issues.apache.org/jira/browse/HBASE-4987 Project: HBase Issue Type: Bug Reporter: Prakash Khemani Assignee: Prakash Khemani Attachments: HBASE-4987.D675.1.patch, HBASE-4987.D675.2.patch @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4120) isolation and allocation
[
https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165857#comment-13165857
]
Ted Yu commented on HBASE-4120:
---
Please add license to QosRegionObserver.java
I think we should enhance ScannerListener.leaseExpired() with
pre/postScannerClose() so that features such as table priority can receive
consistent notification.
The trick here is that RegionScanner only exposes HRegionInfo. We should be
able to utilize onlineRegions in looking up HRegion by region name.
Then we should be able to call the following:
{code}
if (region != null region.getCoprocessorHost() != null) {
region.getCoprocessorHost().postScannerClose(s);
}
{code}
isolation and allocation
Key: HBASE-4120
URL: https://issues.apache.org/jira/browse/HBASE-4120
Project: HBase
Issue Type: New Feature
Components: master, regionserver
Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0
Reporter: Liu Jia
Assignee: Liu Jia
Fix For: 0.94.0
Attachments: Design_document_for_HBase_isolation_and_allocation.pdf,
Design_document_for_HBase_isolation_and_allocation_Revised.pdf,
HBase_isolation_and_allocation_user_guide.pdf,
Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch,
TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v8.patch,
TablePriority_v8.patch, TablePriority_v8_for_trunk.patch,
TablePrioriy_v9.patch
The HBase isolation and allocation tool is designed to help users manage
cluster resource among different application and tables.
When we have a large scale of HBase cluster with many applications running on
it, there will be lots of problems. In Taobao there is a cluster for many
departments to test their applications performance, these applications are
based on HBase. With one cluster which has 12 servers, there will be only one
application running exclusively on this server, and many other applications
must wait until the previous test finished.
After we add allocation manage function to the cluster, applications can
share the cluster and run concurrently. Also if the Test Engineer wants to
make sure there is no interference, he/she can move out other tables from
this group.
In groups we use table priority to allocate resource, when system is busy; we
can make sure high-priority tables are not affected lower-priority tables
Different groups can have different region server configurations, some groups
optimized for reading can have large block cache size, and others optimized
for writing can have large memstore size.
Tables and region servers can be moved easily between groups; after changing
the configuration, a group can be restarted alone instead of restarting the
whole cluster.
git entry : https://github.com/ICT-Ope/HBase_allocation .
We hope our work is helpful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4120: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504706/TablePriority_v8.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -143 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 88 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestRestartCluster org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase org.apache.hadoop.hbase.master.TestMasterFailover org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/330//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/330//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/330//console This message is automatically generated.) isolation and allocation Key: HBASE-4120 URL: https://issues.apache.org/jira/browse/HBASE-4120 Project: HBase Issue Type: New Feature Components: master, regionserver Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, Design_document_for_HBase_isolation_and_allocation_Revised.pdf, HBase_isolation_and_allocation_user_guide.pdf, Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v8.patch, TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables. When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished. After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group. In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables Different groups can have different region server configurations, some groups optimized for reading can have large block cache size, and others optimized for writing can have large memstore size. Tables and region servers can be moved easily between groups; after changing the configuration, a group can be restarted alone instead of restarting the whole cluster. git entry : https://github.com/ICT-Ope/HBase_allocation . We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4120: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504702/TablePriority_v8.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -143 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 88 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap org.apache.hadoop.hbase.master.TestMasterFailover org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase org.apache.hadoop.hbase.master.TestRestartCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/328//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/328//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/328//console This message is automatically generated.) isolation and allocation Key: HBASE-4120 URL: https://issues.apache.org/jira/browse/HBASE-4120 Project: HBase Issue Type: New Feature Components: master, regionserver Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, Design_document_for_HBase_isolation_and_allocation_Revised.pdf, HBase_isolation_and_allocation_user_guide.pdf, Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v8.patch, TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables. When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished. After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group. In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables Different groups can have different region server configurations, some groups optimized for reading can have large block cache size, and others optimized for writing can have large memstore size. Tables and region servers can be moved easily between groups; after changing the configuration, a group can be restarted alone instead of restarting the whole cluster. git entry : https://github.com/ICT-Ope/HBase_allocation . We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4120: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12502140/TablePriority_v8_for_trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -145 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 68 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestStoreFileBlockCacheSummary org.apache.hadoop.hbase.master.TestDistributedLogSplitting Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/154//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/154//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/154//console This message is automatically generated.) isolation and allocation Key: HBASE-4120 URL: https://issues.apache.org/jira/browse/HBASE-4120 Project: HBase Issue Type: New Feature Components: master, regionserver Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, Design_document_for_HBase_isolation_and_allocation_Revised.pdf, HBase_isolation_and_allocation_user_guide.pdf, Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v8.patch, TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables. When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished. After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group. In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables Different groups can have different region server configurations, some groups optimized for reading can have large block cache size, and others optimized for writing can have large memstore size. Tables and region servers can be moved easily between groups; after changing the configuration, a group can be restarted alone instead of restarting the whole cluster. git entry : https://github.com/ICT-Ope/HBase_allocation . We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165861#comment-13165861 ] Hadoop QA commented on HBASE-4987: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12506701/HBASE-4987.D675.2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -160 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 74 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/478//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/478//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/478//console This message is automatically generated. wrong use of incarnation var in SplitLogManager --- Key: HBASE-4987 URL: https://issues.apache.org/jira/browse/HBASE-4987 Project: HBase Issue Type: Bug Reporter: Prakash Khemani Assignee: Prakash Khemani Attachments: HBASE-4987.D675.1.patch, HBASE-4987.D675.2.patch @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to d
[
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165863#comment-13165863
]
stack commented on HBASE-4946:
--
This belongs in 0.92 too?
HTable.coprocessorExec (and possibly coprocessorProxy) does not work with
dynamically loaded coprocessors (from hdfs or local system), because the RPC
system tries to deserialize an unknown class.
-
Key: HBASE-4946
URL: https://issues.apache.org/jira/browse/HBASE-4946
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.0
Reporter: Andrei Dragomir
Assignee: Andrei Dragomir
Attachments: 4946-v4.txt, 4946-v5.txt, HBASE-4946-v2.patch,
HBASE-4946-v3.patch, HBASE-4946.patch
Loading coprocessors jars from hdfs works fine. I load it from the shell,
after setting the attribute, and it gets loaded:
{noformat}
INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor
config now ...
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class
com.MyCoprocessorClass needs to be loaded from a file -
hdfs://localhost:9000/coproc/rt- 0.0.1-SNAPSHOT.jar.
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance:
com.MyCoprocessorClass
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost:
RegionEnvironment createEnvironment
DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol
handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f.
protocol=com.MyCoprocessorClassProtocol
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load
coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
{noformat}
The problem is that this coprocessors simply extends BaseEndpointCoprocessor,
with a dynamic method. When calling this method from the client with
HTable.coprocessorExec, I get errors on the HRegionServer, because the call
cannot be deserialized from writables.
The problem is that Exec tries to do an early resolve of the coprocessor
class. The coprocessor class is loaded, but it is in the context of the
HRegionServer / HRegion. So, the call fails:
{noformat}
2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable:
Error in readFields
java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
at
org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
... 10 more
{noformat}
Probably the correct way to fix this is to make Exec really smart, so that it
knows all the class definitions loaded in CoprocessorHost(s).
I created a small patch that simply doesn't resolve the class definition in
the Exec, instead passing it as string down to the HRegion layer. This layer
knows all the definitions, and simply loads it by name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see:
[jira] [Updated] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to des
[
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-4946:
--
Fix Version/s: 0.94.0
0.92.0
HTable.coprocessorExec (and possibly coprocessorProxy) does not work with
dynamically loaded coprocessors (from hdfs or local system), because the RPC
system tries to deserialize an unknown class.
-
Key: HBASE-4946
URL: https://issues.apache.org/jira/browse/HBASE-4946
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.0
Reporter: Andrei Dragomir
Assignee: Andrei Dragomir
Fix For: 0.92.0, 0.94.0
Attachments: 4946-v4.txt, 4946-v5.txt, HBASE-4946-v2.patch,
HBASE-4946-v3.patch, HBASE-4946.patch
Loading coprocessors jars from hdfs works fine. I load it from the shell,
after setting the attribute, and it gets loaded:
{noformat}
INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor
config now ...
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class
com.MyCoprocessorClass needs to be loaded from a file -
hdfs://localhost:9000/coproc/rt- 0.0.1-SNAPSHOT.jar.
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance:
com.MyCoprocessorClass
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost:
RegionEnvironment createEnvironment
DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol
handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f.
protocol=com.MyCoprocessorClassProtocol
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load
coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
{noformat}
The problem is that this coprocessors simply extends BaseEndpointCoprocessor,
with a dynamic method. When calling this method from the client with
HTable.coprocessorExec, I get errors on the HRegionServer, because the call
cannot be deserialized from writables.
The problem is that Exec tries to do an early resolve of the coprocessor
class. The coprocessor class is loaded, but it is in the context of the
HRegionServer / HRegion. So, the call fails:
{noformat}
2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable:
Error in readFields
java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
at
org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
... 10 more
{noformat}
Probably the correct way to fix this is to make Exec really smart, so that it
knows all the class definitions loaded in CoprocessorHost(s).
I created a small patch that simply doesn't resolve the class definition in
the Exec, instead passing it as string down to the HRegion layer. This layer
knows all the definitions, and simply loads it by name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see:
[jira] [Commented] (HBASE-4980) Null pointer exception in HBaseClient receiveResponse
[
https://issues.apache.org/jira/browse/HBASE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165864#comment-13165864
]
Hudson commented on HBASE-4980:
---
Integrated in HBase-0.92 #179 (See
[https://builds.apache.org/job/HBase-0.92/179/])
HBASE-4980 Null pointer exception in HBaseClient receiveResponse
HBASE-4980 Null pointer exception in HBaseClient receiveResponse
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
stack :
Files :
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
Null pointer exception in HBaseClient receiveResponse
-
Key: HBASE-4980
URL: https://issues.apache.org/jira/browse/HBASE-4980
Project: HBase
Issue Type: Bug
Components: client
Affects Versions: 0.92.0
Reporter: Shrijeet Paliwal
Assignee: Shrijeet Paliwal
Labels: newbie
Fix For: 0.92.0
Attachments:
0001-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
0002-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
Relevant Stack trace:
2011-11-30 13:10:26,557 [IPC Client (47) connection to
xx.xx.xx/172.22.4.68:60020 from an unknown user] WARN
org.apache.hadoop.ipc.HBaseClient - Unexpected exception receiving call
responses
java.lang.NullPointerException
-at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:583)
-at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:511)
{code}
if (LOG.isDebugEnabled())
LOG.debug(getName() + got value # + id);
Call call = calls.remove(id);
// Read the flag byte
byte flag = in.readByte();
boolean isError = ResponseFlag.isError(flag);
if (ResponseFlag.isLength(flag)) {
// Currently length if present is unused.
in.readInt();
}
int state = in.readInt(); // Read the state. Currently unused.
if (isError) {
//noinspection ThrowableInstanceNeverThrown
call.setException(new RemoteException( WritableUtils.readString(in),
WritableUtils.readString(in)));
} else {
{code}
This line {code}Call call = calls.remove(id);{code} may return a null
'call'. It is so because if you have rpc timeout enable, we proactively clean
up other calls which have expired their lifetime along with the call for
which socket timeout exception happend.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to d
[
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165867#comment-13165867
]
Hadoop QA commented on HBASE-4946:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12506703/4946-v5.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -160 warning
messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 75 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/479//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/479//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/479//console
This message is automatically generated.
HTable.coprocessorExec (and possibly coprocessorProxy) does not work with
dynamically loaded coprocessors (from hdfs or local system), because the RPC
system tries to deserialize an unknown class.
-
Key: HBASE-4946
URL: https://issues.apache.org/jira/browse/HBASE-4946
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.0
Reporter: Andrei Dragomir
Assignee: Andrei Dragomir
Fix For: 0.92.0, 0.94.0
Attachments: 4946-v4.txt, 4946-v5.txt, HBASE-4946-v2.patch,
HBASE-4946-v3.patch, HBASE-4946.patch
Loading coprocessors jars from hdfs works fine. I load it from the shell,
after setting the attribute, and it gets loaded:
{noformat}
INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor
config now ...
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class
com.MyCoprocessorClass needs to be loaded from a file -
hdfs://localhost:9000/coproc/rt- 0.0.1-SNAPSHOT.jar.
INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance:
com.MyCoprocessorClass
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost:
RegionEnvironment createEnvironment
DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol
handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f.
protocol=com.MyCoprocessorClassProtocol
INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load
coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
{noformat}
The problem is that this coprocessors simply extends BaseEndpointCoprocessor,
with a dynamic method. When calling this method from the client with
HTable.coprocessorExec, I get errors on the HRegionServer, because the call
cannot be deserialized from writables.
The problem is that Exec tries to do an early resolve of the coprocessor
class. The coprocessor class is loaded, but it is in the context of the
HRegionServer / HRegion. So, the call fails:
{noformat}
2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable:
Error in readFields
java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
at
org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at
