[jira] [Created] (HBASE-8741) Mutations on Regions in recovery mode might have same sequenceIDs
Himanshu Vashishtha created HBASE-8741: -- Summary: Mutations on Regions in recovery mode might have same sequenceIDs Key: HBASE-8741 URL: https://issues.apache.org/jira/browse/HBASE-8741 Project: HBase Issue Type: Bug Components: MTTR Affects Versions: 0.95.1 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Currently, when opening a region, we find the maximum sequence ID from all its HFiles and then set the LogSequenceId of the log (in case the later is at a small value). This works good in recovered.edits case as we are not writing to the region until we have replayed all of its previous edits. With distributed log replay, if we want to enable writes while a region is under recovery, we need to make sure that the logSequenceId > maximum logSequenceId of the old regionserver. Otherwise, we might have a situation where new edits have same (or smaller) sequenceIds. We can store region level information in the WALTrailer, than this scenario could be avoided by: a) reading the trailer of the "last completed" file, i.e., last wal file which has a trailer and, b) completely reading the last wal file (this file would not have the trailer, so it needs to be read completely). In future, if we switch to multi wal file, we could read the trailer for all completed WAL files, and reading the remaining incomplete files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8702) Make WALEditCodec pluggable
[ https://issues.apache.org/jira/browse/HBASE-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681903#comment-13681903 ] Hudson commented on HBASE-8702: --- Integrated in HBase-0.94 #1011 (See [https://builds.apache.org/job/HBase-0.94/1011/]) HBASE-8702: Make WALEditCodec pluggable (Revision 1492518) Result = SUCCESS jyates : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogReader.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogWriter.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEdit.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEditCodec.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationHLogReaderManager.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/FaultySequenceFileLogReader.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCustomWALEditCodec.java > Make WALEditCodec pluggable > --- > > Key: HBASE-8702 > URL: https://issues.apache.org/jira/browse/HBASE-8702 > Project: HBase > Issue Type: Improvement > Components: Replication, wal >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.98.0, 0.95.2, 0.94.9 > > Attachments: hbase-8702-0.94-v0.patch, hbase-8702-0.94-v1.patch, > hbase-8702-trunk-v0.patch, hbase-8702-trunk-v1.patch, > hbase-8702-trunk-v2.patch > > > WALEditCode needs to be pluggable to support alternative serialziation > mechanisms. > The open question here is whether to support the alternative codec when doing > replication - both clusters would need the codec on the classpath, which has > additional overhead and also will be a little bit complicated when making the > WAL serialization backwards compatible in 0.94. > This is the follow-up to HBASE-8636. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4811) Support reverse Scan
[ https://issues.apache.org/jira/browse/HBASE-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-4811: Attachment: hbase-4811-trunkv11.patch Addressing review comments and adjust class/method structure a little > Support reverse Scan > > > Key: HBASE-4811 > URL: https://issues.apache.org/jira/browse/HBASE-4811 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.20.6, 0.94.7 >Reporter: John Carrino >Assignee: Liang Xie > Attachments: 4811-trunk-v10.txt, 4811-trunk-v5.patch, > HBase-4811-0.94.3modified.txt, HBase-4811-0.94-v2.txt, > hbase-4811-trunkv11.patch, hbase-4811-trunkv1.patch, > hbase-4811-trunkv4.patch, hbase-4811-trunkv6.patch, hbase-4811-trunkv7.patch, > hbase-4811-trunkv8.patch, hbase-4811-trunkv9.patch > > > All the documentation I find about HBase says that if you want forward and > reverse scans you should just build 2 tables and one be ascending and one > descending. Is there a fundamental reason that HBase only supports forward > Scan? It seems like a lot of extra space overhead and coding overhead (to > keep them in sync) to support 2 tables. > I am assuming this has been discussed before, but I can't find the > discussions anywhere about it or why it would be infeasible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8702) Make WALEditCodec pluggable
[ https://issues.apache.org/jira/browse/HBASE-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681900#comment-13681900 ] Hudson commented on HBASE-8702: --- Integrated in HBase-0.94-security #165 (See [https://builds.apache.org/job/HBase-0.94-security/165/]) HBASE-8702: Make WALEditCodec pluggable (Revision 1492518) Result = SUCCESS jyates : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogReader.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogWriter.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEdit.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEditCodec.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationHLogReaderManager.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/FaultySequenceFileLogReader.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCustomWALEditCodec.java > Make WALEditCodec pluggable > --- > > Key: HBASE-8702 > URL: https://issues.apache.org/jira/browse/HBASE-8702 > Project: HBase > Issue Type: Improvement > Components: Replication, wal >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.98.0, 0.95.2, 0.94.9 > > Attachments: hbase-8702-0.94-v0.patch, hbase-8702-0.94-v1.patch, > hbase-8702-trunk-v0.patch, hbase-8702-trunk-v1.patch, > hbase-8702-trunk-v2.patch > > > WALEditCode needs to be pluggable to support alternative serialziation > mechanisms. > The open question here is whether to support the alternative codec when doing > replication - both clusters would need the codec on the classpath, which has > additional overhead and also will be a little bit complicated when making the > WAL serialization backwards compatible in 0.94. > This is the follow-up to HBASE-8636. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8603) Backport HBASE-6921 to 0.94
[
https://issues.apache.org/jira/browse/HBASE-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681893#comment-13681893
]
Jesse Yates commented on HBASE-8603:
Hmmm, I'm getting a repeatable failure for this on TestMultiSlaveReplication on
my OSX laptop running java 7, particularly, here is the output from mvn
-version:
{quote}
Apache Maven 3.0.4 (r1232337; 2012-01-17 00:44:56-0800)
Maven home: /Users/jyates/local/apache-maven-3.0.4
Java version: 1.7.0_10, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.7.0_10.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.7.5", arch: "x86_64", family: "mac"
{quote}
The failure is:
{quote}
testMultiSlaveReplication(org.apache.hadoop.hbase.replication.TestMultiSlaveReplication):
Waited too much time for put replication
{quote}
but doesn't have any telling stack traces in the logs.
Tests pass on tip of 0.94 on my linux box though (haven't tried if there is a
fix between tip and now).
Any thoughts Ted?
> Backport HBASE-6921 to 0.94
> ---
>
> Key: HBASE-8603
> URL: https://issues.apache.org/jira/browse/HBASE-8603
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.94.9
>
> Attachments: 8603-0.94.txt
>
>
> String and ConcurrentHashMap sizes change on jdk7
> This was brought up by discussion from HBASE-8014:
> https://issues.apache.org/jira/browse/HBASE-8014?focusedCommentId=13665711&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13665711
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4811) Support reverse Scan
[ https://issues.apache.org/jira/browse/HBASE-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681891#comment-13681891 ] chunhui shen commented on HBASE-4811: - bq.So would: ReversedRegionScanner -> KVHelp -> RerversedStoreScanner -> ReversedKVHeap -> (MemstoreScanner|StoreFileScanner) work? Need ReversedRegionScanner -> ReversedKVHeap, because we need a reversed sort of scanners in ReversedKVHeap and call seekToPreviousRow for RegionScanner#nextRow, otherwise the scan order would be wrong. bq.ReversedKeyValueHeap should not need backwardSeek, since all calls to (re)seek, etc, call backwardSeek anyway. ReversedKeyValueHeap should support backwardSeek and seekToPreviousRow even if no calls. Is it better to understand that prohibit calling (re)seek in ReversedKeyValueHeap and ReversedStoreScanner bq.do we need NonReversedNonLazyKeyValueScanner? Could add "unsupported" implementations for these methods to NonLazyKeyValueScanner. MemStoreScanner extends from NonLazyKeyValueScanner, it seems not appropriate if add "unsupported" implementations to NonLazyKeyValueScanner > Support reverse Scan > > > Key: HBASE-4811 > URL: https://issues.apache.org/jira/browse/HBASE-4811 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.20.6, 0.94.7 >Reporter: John Carrino >Assignee: Liang Xie > Attachments: 4811-trunk-v10.txt, 4811-trunk-v5.patch, > HBase-4811-0.94.3modified.txt, HBase-4811-0.94-v2.txt, > hbase-4811-trunkv1.patch, hbase-4811-trunkv4.patch, hbase-4811-trunkv6.patch, > hbase-4811-trunkv7.patch, hbase-4811-trunkv8.patch, hbase-4811-trunkv9.patch > > > All the documentation I find about HBase says that if you want forward and > reverse scans you should just build 2 tables and one be ascending and one > descending. Is there a fundamental reason that HBase only supports forward > Scan? It seems like a lot of extra space overhead and coding overhead (to > keep them in sync) to support 2 tables. > I am assuming this has been discussed before, but I can't find the > discussions anywhere about it or why it would be infeasible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8740) Generate unique table names in TestAccessController
[ https://issues.apache.org/jira/browse/HBASE-8740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681884#comment-13681884 ] Jesse Yates commented on HBASE-8740: [~andrew.purt...@gmail.com] my thoughts exactly > Generate unique table names in TestAccessController > --- > > Key: HBASE-8740 > URL: https://issues.apache.org/jira/browse/HBASE-8740 > Project: HBase > Issue Type: Improvement >Reporter: Alex Araujo >Priority: Minor > Attachments: HBASE-8740.patch, HBASE-8740v2.patch > > > This test class creates/drops a table named 'testtable' for 30+ tests. > Having tables named after each test method may help diagnose test failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8740) Generate unique table names in TestAccessController
[ https://issues.apache.org/jira/browse/HBASE-8740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681880#comment-13681880 ] Andrew Purtell commented on HBASE-8740: --- +1 And there are many other places we could use this new @Rule. > Generate unique table names in TestAccessController > --- > > Key: HBASE-8740 > URL: https://issues.apache.org/jira/browse/HBASE-8740 > Project: HBase > Issue Type: Improvement >Reporter: Alex Araujo >Priority: Minor > Attachments: HBASE-8740.patch, HBASE-8740v2.patch > > > This test class creates/drops a table named 'testtable' for 30+ tests. > Having tables named after each test method may help diagnose test failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8702) Make WALEditCodec pluggable
[ https://issues.apache.org/jira/browse/HBASE-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681833#comment-13681833 ] Hudson commented on HBASE-8702: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #566 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/566/]) HBASE-8702: Make WALEditCodec pluggable (Revision 1492407) Result = FAILURE jyates : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogReader.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCustomWALCellCodec.java > Make WALEditCodec pluggable > --- > > Key: HBASE-8702 > URL: https://issues.apache.org/jira/browse/HBASE-8702 > Project: HBase > Issue Type: Improvement > Components: Replication, wal >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.98.0, 0.95.2, 0.94.9 > > Attachments: hbase-8702-0.94-v0.patch, hbase-8702-0.94-v1.patch, > hbase-8702-trunk-v0.patch, hbase-8702-trunk-v1.patch, > hbase-8702-trunk-v2.patch > > > WALEditCode needs to be pluggable to support alternative serialziation > mechanisms. > The open question here is whether to support the alternative codec when doing > replication - both clusters would need the codec on the classpath, which has > additional overhead and also will be a little bit complicated when making the > WAL serialization backwards compatible in 0.94. > This is the follow-up to HBASE-8636. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8706) Some improvement in snapshot
[
https://issues.apache.org/jira/browse/HBASE-8706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681832#comment-13681832
]
Hudson commented on HBASE-8706:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #566 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/566/])
HBASE-8706 Some improvement in snapshot (binlijin) (Revision 1492398)
Result = FAILURE
mbertozzi :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureCoordinator.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureMember.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/Subprocedure.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/snapshot/RegionServerSnapshotManager.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureCoordinator.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureMember.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedure.java
> Some improvement in snapshot
>
>
> Key: HBASE-8706
> URL: https://issues.apache.org/jira/browse/HBASE-8706
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.94.8, 0.95.0
>Reporter: binlijin
>Assignee: Matteo Bertozzi
> Fix For: 0.98.0, 0.95.2
>
> Attachments: HBASE-8706-2.patch, HBASE-8706-3.patch,
> HBASE-8706.patch, HBASE-8706-v4.patch, HBASE-8706-v4.patch
>
>
> (1)timeout for Procedure can not be configured.
> {code}
> Procedure's timeout
> ProcedureCoordinator
> final static long TIMEOUT_MILLIS_DEFAULT = 6;
>createProcedure(ForeignExceptionDispatcher fed, String procName, byte[]
> procArgs,
> List expectedMembers) {
> // build the procedure
> return new Procedure(this, fed, WAKE_MILLIS_DEFAULT,
> TIMEOUT_MILLIS_DEFAULT,
> procName, procArgs, expectedMembers);
> }
> RegionServerSnapshotManager:
> /** Conf key for max time to keep threads in snapshot request pool waiting
> */
> public static final String SNAPSHOT_TIMEOUT_MILLIS_KEY =
> "hbase.snapshot.region.timeout";
> /** Keep threads alive in request pool for max of 60 seconds */
> public static final long SNAPSHOT_TIMEOUT_MILLIS_DEFAULT = 6;
> public Subprocedure buildSubprocedure(SnapshotDescription snapshot) {
> long timeoutMillis = conf.getLong(SNAPSHOT_TIMEOUT_MILLIS_KEY,
> SNAPSHOT_TIMEOUT_MILLIS_DEFAULT);
> case FLUSH:
> SnapshotSubprocedurePool taskManager =
> new SnapshotSubprocedurePool(rss.getServerName().toString(), conf);
> }
> {code}
> (2)TakeSnapshotHandler
> after snapshotRegions we should call monitor.rethrowException(); to check if
> there is exception and if there is we can skip the verifySnapshot
> (3)too much error message when error happened in some place.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8738) [refGuide] Overhauled HBase metrics section
[ https://issues.apache.org/jira/browse/HBASE-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681834#comment-13681834 ] Hudson commented on HBASE-8738: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #566 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/566/]) hbase-8738. refguide, OpsMgt chapter, overhauled HBase metrics section. (Revision 1492439) Result = FAILURE > [refGuide] Overhauled HBase metrics section > --- > > Key: HBASE-8738 > URL: https://issues.apache.org/jira/browse/HBASE-8738 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil > Attachments: ops_mgt_hbase_8738.xml.patch > > > Overhauled HBase metrics section in Ops chapter. > * Broke out flat list of metrics into 2 sub-sections: most important, and > "other". > * Added many metrics, and improved descriptions of all metrics. > * Added warning to Ganglia users about default metric emission. > Thanks to Elliot Clark, Greg Waffen, James McDermott, Charles Douthart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8739) [refGuide] corrected map-task "spitting"
[ https://issues.apache.org/jira/browse/HBASE-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681835#comment-13681835 ] Hudson commented on HBASE-8739: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #566 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/566/]) hbase-8739. book.xml fixed typo in MapReduce chapter "spitting" (Revision 1492455) Result = FAILURE > [refGuide] corrected map-task "spitting" > > > Key: HBASE-8739 > URL: https://issues.apache.org/jira/browse/HBASE-8739 > Project: HBase > Issue Type: Bug >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Trivial > Attachments: book_hbase_8739.xml.patch > > > Nit fix in the MapReduce chapter. > Corrected section heading "Map-Task Spitting" to "Map-Task Splitting". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8702) Make WALEditCodec pluggable
[ https://issues.apache.org/jira/browse/HBASE-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681827#comment-13681827 ] Jesse Yates commented on HBASE-8702: Looks like the 0.95 issues are timeout related...should be good to go. > Make WALEditCodec pluggable > --- > > Key: HBASE-8702 > URL: https://issues.apache.org/jira/browse/HBASE-8702 > Project: HBase > Issue Type: Improvement > Components: Replication, wal >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.98.0, 0.95.2, 0.94.9 > > Attachments: hbase-8702-0.94-v0.patch, hbase-8702-0.94-v1.patch, > hbase-8702-trunk-v0.patch, hbase-8702-trunk-v1.patch, > hbase-8702-trunk-v2.patch > > > WALEditCode needs to be pluggable to support alternative serialziation > mechanisms. > The open question here is whether to support the alternative codec when doing > replication - both clusters would need the codec on the classpath, which has > additional overhead and also will be a little bit complicated when making the > WAL serialization backwards compatible in 0.94. > This is the follow-up to HBASE-8636. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8702) Make WALEditCodec pluggable
[ https://issues.apache.org/jira/browse/HBASE-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681807#comment-13681807 ] Hudson commented on HBASE-8702: --- Integrated in HBase-TRUNK #4177 (See [https://builds.apache.org/job/HBase-TRUNK/4177/]) HBASE-8702: Make WALEditCodec pluggable (Revision 1492407) Result = SUCCESS jyates : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogReader.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCustomWALCellCodec.java > Make WALEditCodec pluggable > --- > > Key: HBASE-8702 > URL: https://issues.apache.org/jira/browse/HBASE-8702 > Project: HBase > Issue Type: Improvement > Components: Replication, wal >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.98.0, 0.95.2, 0.94.9 > > Attachments: hbase-8702-0.94-v0.patch, hbase-8702-0.94-v1.patch, > hbase-8702-trunk-v0.patch, hbase-8702-trunk-v1.patch, > hbase-8702-trunk-v2.patch > > > WALEditCode needs to be pluggable to support alternative serialziation > mechanisms. > The open question here is whether to support the alternative codec when doing > replication - both clusters would need the codec on the classpath, which has > additional overhead and also will be a little bit complicated when making the > WAL serialization backwards compatible in 0.94. > This is the follow-up to HBASE-8636. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8738) [refGuide] Overhauled HBase metrics section
[ https://issues.apache.org/jira/browse/HBASE-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681808#comment-13681808 ] Hudson commented on HBASE-8738: --- Integrated in HBase-TRUNK #4177 (See [https://builds.apache.org/job/HBase-TRUNK/4177/]) hbase-8738. refguide, OpsMgt chapter, overhauled HBase metrics section. (Revision 1492439) Result = SUCCESS > [refGuide] Overhauled HBase metrics section > --- > > Key: HBASE-8738 > URL: https://issues.apache.org/jira/browse/HBASE-8738 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil > Attachments: ops_mgt_hbase_8738.xml.patch > > > Overhauled HBase metrics section in Ops chapter. > * Broke out flat list of metrics into 2 sub-sections: most important, and > "other". > * Added many metrics, and improved descriptions of all metrics. > * Added warning to Ganglia users about default metric emission. > Thanks to Elliot Clark, Greg Waffen, James McDermott, Charles Douthart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8739) [refGuide] corrected map-task "spitting"
[ https://issues.apache.org/jira/browse/HBASE-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681809#comment-13681809 ] Hudson commented on HBASE-8739: --- Integrated in HBase-TRUNK #4177 (See [https://builds.apache.org/job/HBase-TRUNK/4177/]) hbase-8739. book.xml fixed typo in MapReduce chapter "spitting" (Revision 1492455) Result = SUCCESS > [refGuide] corrected map-task "spitting" > > > Key: HBASE-8739 > URL: https://issues.apache.org/jira/browse/HBASE-8739 > Project: HBase > Issue Type: Bug >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Trivial > Attachments: book_hbase_8739.xml.patch > > > Nit fix in the MapReduce chapter. > Corrected section heading "Map-Task Spitting" to "Map-Task Splitting". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8706) Some improvement in snapshot
[
https://issues.apache.org/jira/browse/HBASE-8706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681794#comment-13681794
]
Hudson commented on HBASE-8706:
---
Integrated in hbase-0.95-on-hadoop2 #131 (See
[https://builds.apache.org/job/hbase-0.95-on-hadoop2/131/])
HBASE-8706 Some improvement in snapshot (binlijin) (Revision 1492399)
Result = FAILURE
mbertozzi :
Files :
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureCoordinator.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureMember.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/Subprocedure.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/snapshot/RegionServerSnapshotManager.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureCoordinator.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureMember.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedure.java
> Some improvement in snapshot
>
>
> Key: HBASE-8706
> URL: https://issues.apache.org/jira/browse/HBASE-8706
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.94.8, 0.95.0
>Reporter: binlijin
>Assignee: Matteo Bertozzi
> Fix For: 0.98.0, 0.95.2
>
> Attachments: HBASE-8706-2.patch, HBASE-8706-3.patch,
> HBASE-8706.patch, HBASE-8706-v4.patch, HBASE-8706-v4.patch
>
>
> (1)timeout for Procedure can not be configured.
> {code}
> Procedure's timeout
> ProcedureCoordinator
> final static long TIMEOUT_MILLIS_DEFAULT = 6;
>createProcedure(ForeignExceptionDispatcher fed, String procName, byte[]
> procArgs,
> List expectedMembers) {
> // build the procedure
> return new Procedure(this, fed, WAKE_MILLIS_DEFAULT,
> TIMEOUT_MILLIS_DEFAULT,
> procName, procArgs, expectedMembers);
> }
> RegionServerSnapshotManager:
> /** Conf key for max time to keep threads in snapshot request pool waiting
> */
> public static final String SNAPSHOT_TIMEOUT_MILLIS_KEY =
> "hbase.snapshot.region.timeout";
> /** Keep threads alive in request pool for max of 60 seconds */
> public static final long SNAPSHOT_TIMEOUT_MILLIS_DEFAULT = 6;
> public Subprocedure buildSubprocedure(SnapshotDescription snapshot) {
> long timeoutMillis = conf.getLong(SNAPSHOT_TIMEOUT_MILLIS_KEY,
> SNAPSHOT_TIMEOUT_MILLIS_DEFAULT);
> case FLUSH:
> SnapshotSubprocedurePool taskManager =
> new SnapshotSubprocedurePool(rss.getServerName().toString(), conf);
> }
> {code}
> (2)TakeSnapshotHandler
> after snapshotRegions we should call monitor.rethrowException(); to check if
> there is exception and if there is we can skip the verifySnapshot
> (3)too much error message when error happened in some place.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8702) Make WALEditCodec pluggable
[ https://issues.apache.org/jira/browse/HBASE-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681795#comment-13681795 ] Hudson commented on HBASE-8702: --- Integrated in hbase-0.95-on-hadoop2 #131 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/131/]) HBASE-8702: Make WALEditCodec pluggable (Revision 1492414) Result = FAILURE jyates : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogReader.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCustomWALCellCodec.java > Make WALEditCodec pluggable > --- > > Key: HBASE-8702 > URL: https://issues.apache.org/jira/browse/HBASE-8702 > Project: HBase > Issue Type: Improvement > Components: Replication, wal >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.98.0, 0.95.2, 0.94.9 > > Attachments: hbase-8702-0.94-v0.patch, hbase-8702-0.94-v1.patch, > hbase-8702-trunk-v0.patch, hbase-8702-trunk-v1.patch, > hbase-8702-trunk-v2.patch > > > WALEditCode needs to be pluggable to support alternative serialziation > mechanisms. > The open question here is whether to support the alternative codec when doing > replication - both clusters would need the codec on the classpath, which has > additional overhead and also will be a little bit complicated when making the > WAL serialization backwards compatible in 0.94. > This is the follow-up to HBASE-8636. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8705) RS holding META when restarted in a single node setup may hang infinitely without META assignment
[
https://issues.apache.org/jira/browse/HBASE-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-8705:
--
Status: Patch Available (was: Open)
> RS holding META when restarted in a single node setup may hang infinitely
> without META assignment
> -
>
> Key: HBASE-8705
> URL: https://issues.apache.org/jira/browse/HBASE-8705
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.1
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 0.98.0
>
> Attachments: HBASE-8705_1.patch, HBASE-8705.patch
>
>
> This bug may be minor as it likely to happen in a single node setup.
> I restarted the RS holding META. The master tried assigning META using
> MetaSSH. But tried this before the new RS came up.
> So as not region plan is found
> {code}
> if (plan == null) {
> LOG.warn("Unable to determine a plan to assign " + region);
> if (tomActivated){
> this.timeoutMonitor.setAllRegionServersOffline(true);
> } else {
> regionStates.updateRegionState(region,
> RegionState.State.FAILED_OPEN);
> }
> return;
> }
> {code}
> we just return without assigment. And this being the META the small cluster
> just hangs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8705) RS holding META when restarted in a single node setup may hang infinitely without META assignment
[
https://issues.apache.org/jira/browse/HBASE-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-8705:
--
Attachment: HBASE-8705_1.patch
Updated patch that resets the interrupt flag.
Also removed the return stmt from InterruptedException catch block. The stmt
below that would be enough.
Thanks for the reviews.
> RS holding META when restarted in a single node setup may hang infinitely
> without META assignment
> -
>
> Key: HBASE-8705
> URL: https://issues.apache.org/jira/browse/HBASE-8705
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.1
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 0.98.0
>
> Attachments: HBASE-8705_1.patch, HBASE-8705.patch
>
>
> This bug may be minor as it likely to happen in a single node setup.
> I restarted the RS holding META. The master tried assigning META using
> MetaSSH. But tried this before the new RS came up.
> So as not region plan is found
> {code}
> if (plan == null) {
> LOG.warn("Unable to determine a plan to assign " + region);
> if (tomActivated){
> this.timeoutMonitor.setAllRegionServersOffline(true);
> } else {
> regionStates.updateRegionState(region,
> RegionState.State.FAILED_OPEN);
> }
> return;
> }
> {code}
> we just return without assigment. And this being the META the small cluster
> just hangs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8705) RS holding META when restarted in a single node setup may hang infinitely without META assignment
[
https://issues.apache.org/jira/browse/HBASE-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-8705:
--
Status: Open (was: Patch Available)
> RS holding META when restarted in a single node setup may hang infinitely
> without META assignment
> -
>
> Key: HBASE-8705
> URL: https://issues.apache.org/jira/browse/HBASE-8705
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.1
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 0.98.0
>
> Attachments: HBASE-8705_1.patch, HBASE-8705.patch
>
>
> This bug may be minor as it likely to happen in a single node setup.
> I restarted the RS holding META. The master tried assigning META using
> MetaSSH. But tried this before the new RS came up.
> So as not region plan is found
> {code}
> if (plan == null) {
> LOG.warn("Unable to determine a plan to assign " + region);
> if (tomActivated){
> this.timeoutMonitor.setAllRegionServersOffline(true);
> } else {
> regionStates.updateRegionState(region,
> RegionState.State.FAILED_OPEN);
> }
> return;
> }
> {code}
> we just return without assigment. And this being the META the small cluster
> just hangs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8706) Some improvement in snapshot
[
https://issues.apache.org/jira/browse/HBASE-8706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681752#comment-13681752
]
Hudson commented on HBASE-8706:
---
Integrated in hbase-0.95 #241 (See
[https://builds.apache.org/job/hbase-0.95/241/])
HBASE-8706 Some improvement in snapshot (binlijin) (Revision 1492399)
Result = FAILURE
mbertozzi :
Files :
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureCoordinator.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureMember.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/Subprocedure.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/snapshot/RegionServerSnapshotManager.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureCoordinator.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureMember.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedure.java
> Some improvement in snapshot
>
>
> Key: HBASE-8706
> URL: https://issues.apache.org/jira/browse/HBASE-8706
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.94.8, 0.95.0
>Reporter: binlijin
>Assignee: Matteo Bertozzi
> Fix For: 0.98.0, 0.95.2
>
> Attachments: HBASE-8706-2.patch, HBASE-8706-3.patch,
> HBASE-8706.patch, HBASE-8706-v4.patch, HBASE-8706-v4.patch
>
>
> (1)timeout for Procedure can not be configured.
> {code}
> Procedure's timeout
> ProcedureCoordinator
> final static long TIMEOUT_MILLIS_DEFAULT = 6;
>createProcedure(ForeignExceptionDispatcher fed, String procName, byte[]
> procArgs,
> List expectedMembers) {
> // build the procedure
> return new Procedure(this, fed, WAKE_MILLIS_DEFAULT,
> TIMEOUT_MILLIS_DEFAULT,
> procName, procArgs, expectedMembers);
> }
> RegionServerSnapshotManager:
> /** Conf key for max time to keep threads in snapshot request pool waiting
> */
> public static final String SNAPSHOT_TIMEOUT_MILLIS_KEY =
> "hbase.snapshot.region.timeout";
> /** Keep threads alive in request pool for max of 60 seconds */
> public static final long SNAPSHOT_TIMEOUT_MILLIS_DEFAULT = 6;
> public Subprocedure buildSubprocedure(SnapshotDescription snapshot) {
> long timeoutMillis = conf.getLong(SNAPSHOT_TIMEOUT_MILLIS_KEY,
> SNAPSHOT_TIMEOUT_MILLIS_DEFAULT);
> case FLUSH:
> SnapshotSubprocedurePool taskManager =
> new SnapshotSubprocedurePool(rss.getServerName().toString(), conf);
> }
> {code}
> (2)TakeSnapshotHandler
> after snapshotRegions we should call monitor.rethrowException(); to check if
> there is exception and if there is we can skip the verifySnapshot
> (3)too much error message when error happened in some place.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8702) Make WALEditCodec pluggable
[ https://issues.apache.org/jira/browse/HBASE-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681753#comment-13681753 ] Hudson commented on HBASE-8702: --- Integrated in hbase-0.95 #241 (See [https://builds.apache.org/job/hbase-0.95/241/]) HBASE-8702: Make WALEditCodec pluggable (Revision 1492414) Result = FAILURE jyates : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogReader.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCustomWALCellCodec.java > Make WALEditCodec pluggable > --- > > Key: HBASE-8702 > URL: https://issues.apache.org/jira/browse/HBASE-8702 > Project: HBase > Issue Type: Improvement > Components: Replication, wal >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.98.0, 0.95.2, 0.94.9 > > Attachments: hbase-8702-0.94-v0.patch, hbase-8702-0.94-v1.patch, > hbase-8702-trunk-v0.patch, hbase-8702-trunk-v1.patch, > hbase-8702-trunk-v2.patch > > > WALEditCode needs to be pluggable to support alternative serialziation > mechanisms. > The open question here is whether to support the alternative codec when doing > replication - both clusters would need the codec on the classpath, which has > additional overhead and also will be a little bit complicated when making the > WAL serialization backwards compatible in 0.94. > This is the follow-up to HBASE-8636. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8740) Generate unique table names in TestAccessController
[ https://issues.apache.org/jira/browse/HBASE-8740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681732#comment-13681732 ] Jesse Yates commented on HBASE-8740: +1 ship it. I know [~saint@gmail.com] wanted to take a look, but I'm happy to do the commit work. > Generate unique table names in TestAccessController > --- > > Key: HBASE-8740 > URL: https://issues.apache.org/jira/browse/HBASE-8740 > Project: HBase > Issue Type: Improvement >Reporter: Alex Araujo >Priority: Minor > Attachments: HBASE-8740.patch, HBASE-8740v2.patch > > > This test class creates/drops a table named 'testtable' for 30+ tests. > Having tables named after each test method may help diagnose test failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8740) Generate unique table names in TestAccessController
[ https://issues.apache.org/jira/browse/HBASE-8740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Araujo updated HBASE-8740: --- Attachment: HBASE-8740v2.patch Moved TEST_TABLE Rule up > Generate unique table names in TestAccessController > --- > > Key: HBASE-8740 > URL: https://issues.apache.org/jira/browse/HBASE-8740 > Project: HBase > Issue Type: Improvement >Reporter: Alex Araujo >Priority: Minor > Attachments: HBASE-8740.patch, HBASE-8740v2.patch > > > This test class creates/drops a table named 'testtable' for 30+ tests. > Having tables named after each test method may help diagnose test failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8740) Generate unique table names in TestAccessController
[
https://issues.apache.org/jira/browse/HBASE-8740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681718#comment-13681718
]
Jesse Yates commented on HBASE-8740:
+1 looks good to me.
Minor nit:
{code}
@Rule public TableName TEST_TABLE = new TableName();
{code}
Maybe move this up out of the regular constants? Its generally better style to
write things somewhat in order that they are run - it makes it a little easier
to understand.
> Generate unique table names in TestAccessController
> ---
>
> Key: HBASE-8740
> URL: https://issues.apache.org/jira/browse/HBASE-8740
> Project: HBase
> Issue Type: Improvement
>Reporter: Alex Araujo
>Priority: Minor
> Attachments: HBASE-8740.patch
>
>
> This test class creates/drops a table named 'testtable' for 30+ tests.
> Having tables named after each test method may help diagnose test failures.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8740) Generate unique table names in TestAccessController
[ https://issues.apache.org/jira/browse/HBASE-8740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Araujo updated HBASE-8740: --- Status: Patch Available (was: Open) Added unique table names to TestAccessController via a JUnit Rule > Generate unique table names in TestAccessController > --- > > Key: HBASE-8740 > URL: https://issues.apache.org/jira/browse/HBASE-8740 > Project: HBase > Issue Type: Improvement >Reporter: Alex Araujo >Priority: Minor > Attachments: HBASE-8740.patch > > > This test class creates/drops a table named 'testtable' for 30+ tests. > Having tables named after each test method may help diagnose test failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8740) Generate unique table names in TestAccessController
[ https://issues.apache.org/jira/browse/HBASE-8740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Araujo updated HBASE-8740: --- Attachment: HBASE-8740.patch > Generate unique table names in TestAccessController > --- > > Key: HBASE-8740 > URL: https://issues.apache.org/jira/browse/HBASE-8740 > Project: HBase > Issue Type: Improvement >Reporter: Alex Araujo >Priority: Minor > Attachments: HBASE-8740.patch > > > This test class creates/drops a table named 'testtable' for 30+ tests. > Having tables named after each test method may help diagnose test failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7462) TestDrainingServer is an integration test. It should be a unit test instead
[ https://issues.apache.org/jira/browse/HBASE-7462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gustavo Anatoly updated HBASE-7462: --- Status: Patch Available (was: Open) > TestDrainingServer is an integration test. It should be a unit test instead > --- > > Key: HBASE-7462 > URL: https://issues.apache.org/jira/browse/HBASE-7462 > Project: HBase > Issue Type: Wish > Components: test >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Priority: Trivial > Labels: noob > Attachments: HBASE-7462-v1.patch > > > TestDrainingServer tests the function that allows to say that a regionserver > should not get new regions. > As it is written today, it's an integration test: it starts & stops a cluster. > The test would be more efficient if it would just check that the > AssignmentManager does not use the drained region server; whatever the > circumstances (bulk assign or not for example). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8740) Generate unique table names in TestAccessController
Alex Araujo created HBASE-8740: -- Summary: Generate unique table names in TestAccessController Key: HBASE-8740 URL: https://issues.apache.org/jira/browse/HBASE-8740 Project: HBase Issue Type: Improvement Reporter: Alex Araujo Priority: Minor This test class creates/drops a table named 'testtable' for 30+ tests. Having tables named after each test method may help diagnose test failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8739) [refGuide] corrected map-task "spitting"
[ https://issues.apache.org/jira/browse/HBASE-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-8739: - Attachment: book_hbase_8739.xml.patch > [refGuide] corrected map-task "spitting" > > > Key: HBASE-8739 > URL: https://issues.apache.org/jira/browse/HBASE-8739 > Project: HBase > Issue Type: Bug >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Trivial > Attachments: book_hbase_8739.xml.patch > > > Nit fix in the MapReduce chapter. > Corrected section heading "Map-Task Spitting" to "Map-Task Splitting". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8739) [refGuide] corrected map-task "spitting"
[ https://issues.apache.org/jira/browse/HBASE-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-8739: - Resolution: Fixed Status: Resolved (was: Patch Available) > [refGuide] corrected map-task "spitting" > > > Key: HBASE-8739 > URL: https://issues.apache.org/jira/browse/HBASE-8739 > Project: HBase > Issue Type: Bug >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Trivial > Attachments: book_hbase_8739.xml.patch > > > Nit fix in the MapReduce chapter. > Corrected section heading "Map-Task Spitting" to "Map-Task Splitting". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8739) [refGuide] corrected map-task "spitting"
[ https://issues.apache.org/jira/browse/HBASE-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-8739: - Status: Patch Available (was: Open) > [refGuide] corrected map-task "spitting" > > > Key: HBASE-8739 > URL: https://issues.apache.org/jira/browse/HBASE-8739 > Project: HBase > Issue Type: Bug >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Trivial > Attachments: book_hbase_8739.xml.patch > > > Nit fix in the MapReduce chapter. > Corrected section heading "Map-Task Spitting" to "Map-Task Splitting". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8739) [refGuide] corrected map-task "spitting"
Doug Meil created HBASE-8739: Summary: [refGuide] corrected map-task "spitting" Key: HBASE-8739 URL: https://issues.apache.org/jira/browse/HBASE-8739 Project: HBase Issue Type: Bug Reporter: Doug Meil Assignee: Doug Meil Priority: Trivial Nit fix in the MapReduce chapter. Corrected section heading "Map-Task Spitting" to "Map-Task Splitting". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7462) TestDrainingServer is an integration test. It should be a unit test instead
[ https://issues.apache.org/jira/browse/HBASE-7462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gustavo Anatoly updated HBASE-7462: --- Attachment: HBASE-7462-v1.patch > TestDrainingServer is an integration test. It should be a unit test instead > --- > > Key: HBASE-7462 > URL: https://issues.apache.org/jira/browse/HBASE-7462 > Project: HBase > Issue Type: Wish > Components: test >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Priority: Trivial > Labels: noob > Attachments: HBASE-7462-v1.patch > > > TestDrainingServer tests the function that allows to say that a regionserver > should not get new regions. > As it is written today, it's an integration test: it starts & stops a cluster. > The test would be more efficient if it would just check that the > AssignmentManager does not use the drained region server; whatever the > circumstances (bulk assign or not for example). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8706) Some improvement in snapshot
[
https://issues.apache.org/jira/browse/HBASE-8706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681673#comment-13681673
]
Hudson commented on HBASE-8706:
---
Integrated in HBase-TRUNK #4176 (See
[https://builds.apache.org/job/HBase-TRUNK/4176/])
HBASE-8706 Some improvement in snapshot (binlijin) (Revision 1492398)
Result = SUCCESS
mbertozzi :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureCoordinator.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureMember.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/Subprocedure.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/snapshot/RegionServerSnapshotManager.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureCoordinator.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureMember.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedure.java
> Some improvement in snapshot
>
>
> Key: HBASE-8706
> URL: https://issues.apache.org/jira/browse/HBASE-8706
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.94.8, 0.95.0
>Reporter: binlijin
>Assignee: Matteo Bertozzi
> Fix For: 0.98.0, 0.95.2
>
> Attachments: HBASE-8706-2.patch, HBASE-8706-3.patch,
> HBASE-8706.patch, HBASE-8706-v4.patch, HBASE-8706-v4.patch
>
>
> (1)timeout for Procedure can not be configured.
> {code}
> Procedure's timeout
> ProcedureCoordinator
> final static long TIMEOUT_MILLIS_DEFAULT = 6;
>createProcedure(ForeignExceptionDispatcher fed, String procName, byte[]
> procArgs,
> List expectedMembers) {
> // build the procedure
> return new Procedure(this, fed, WAKE_MILLIS_DEFAULT,
> TIMEOUT_MILLIS_DEFAULT,
> procName, procArgs, expectedMembers);
> }
> RegionServerSnapshotManager:
> /** Conf key for max time to keep threads in snapshot request pool waiting
> */
> public static final String SNAPSHOT_TIMEOUT_MILLIS_KEY =
> "hbase.snapshot.region.timeout";
> /** Keep threads alive in request pool for max of 60 seconds */
> public static final long SNAPSHOT_TIMEOUT_MILLIS_DEFAULT = 6;
> public Subprocedure buildSubprocedure(SnapshotDescription snapshot) {
> long timeoutMillis = conf.getLong(SNAPSHOT_TIMEOUT_MILLIS_KEY,
> SNAPSHOT_TIMEOUT_MILLIS_DEFAULT);
> case FLUSH:
> SnapshotSubprocedurePool taskManager =
> new SnapshotSubprocedurePool(rss.getServerName().toString(), conf);
> }
> {code}
> (2)TakeSnapshotHandler
> after snapshotRegions we should call monitor.rethrowException(); to check if
> there is exception and if there is we can skip the verifySnapshot
> (3)too much error message when error happened in some place.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8738) [refGuide] Overhauled HBase metrics section
[ https://issues.apache.org/jira/browse/HBASE-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-8738: - Resolution: Fixed Status: Resolved (was: Patch Available) > [refGuide] Overhauled HBase metrics section > --- > > Key: HBASE-8738 > URL: https://issues.apache.org/jira/browse/HBASE-8738 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil > Attachments: ops_mgt_hbase_8738.xml.patch > > > Overhauled HBase metrics section in Ops chapter. > * Broke out flat list of metrics into 2 sub-sections: most important, and > "other". > * Added many metrics, and improved descriptions of all metrics. > * Added warning to Ganglia users about default metric emission. > Thanks to Elliot Clark, Greg Waffen, James McDermott, Charles Douthart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8738) [refGuide] Overhauled HBase metrics section
[ https://issues.apache.org/jira/browse/HBASE-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-8738: - Attachment: ops_mgt_hbase_8738.xml.patch > [refGuide] Overhauled HBase metrics section > --- > > Key: HBASE-8738 > URL: https://issues.apache.org/jira/browse/HBASE-8738 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil > Attachments: ops_mgt_hbase_8738.xml.patch > > > Overhauled HBase metrics section in Ops chapter. > * Broke out flat list of metrics into 2 sub-sections: most important, and > "other". > * Added many metrics, and improved descriptions of all metrics. > * Added warning to Ganglia users about default metric emission. > Thanks to Elliot Clark, Greg Waffen, James McDermott, Charles Douthart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8738) [refGuide] Overhauled HBase metrics section
[ https://issues.apache.org/jira/browse/HBASE-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-8738: - Status: Patch Available (was: Open) > [refGuide] Overhauled HBase metrics section > --- > > Key: HBASE-8738 > URL: https://issues.apache.org/jira/browse/HBASE-8738 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil > Attachments: ops_mgt_hbase_8738.xml.patch > > > Overhauled HBase metrics section in Ops chapter. > * Broke out flat list of metrics into 2 sub-sections: most important, and > "other". > * Added many metrics, and improved descriptions of all metrics. > * Added warning to Ganglia users about default metric emission. > Thanks to Elliot Clark, Greg Waffen, James McDermott, Charles Douthart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8738) [refGuide] Overhauled HBase metrics section
Doug Meil created HBASE-8738: Summary: [refGuide] Overhauled HBase metrics section Key: HBASE-8738 URL: https://issues.apache.org/jira/browse/HBASE-8738 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Overhauled HBase metrics section in Ops chapter. * Broke out flat list of metrics into 2 sub-sections: most important, and "other". * Added many metrics, and improved descriptions of all metrics. * Added warning to Ganglia users about default metric emission. Thanks to Elliot Clark, Greg Waffen, James McDermott, Charles Douthart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8521) Cells cannot be overwritten with bulk loaded HFiles
[
https://issues.apache.org/jira/browse/HBASE-8521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681645#comment-13681645
]
Jean-Marc Spaggiari commented on HBASE-8521:
Any chance to have someone looking at this one?
> Cells cannot be overwritten with bulk loaded HFiles
> ---
>
> Key: HBASE-8521
> URL: https://issues.apache.org/jira/browse/HBASE-8521
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1
>Reporter: Jonathan Natkins
> Attachments: HBASE-8521.diff, HBASE-8521-v0-0.94.patch,
> HBASE-8521-v1-0.94.patch, hfileDirs.tar.gz
>
>
> Let's say you have a pre-built HFile that contains a cell:
> ('rowkey1', 'family1', 'qual1', 1234L, 'value1')
> We bulk load this first HFile. Now, let's create a second HFile that contains
> a cell that overwrites the first:
> ('rowkey1', 'family1', 'qual1', 1234L, 'value2')
> That gets bulk loaded into the table, but the value that HBase bubbles up is
> still 'value1'.
> It seems that there's no way to overwrite a cell for a particular timestamp
> without an explicit put operation. This seems to be the case even after minor
> and major compactions happen.
> My guess is that this is pretty closely related to the sequence number work
> being done on the compaction algorithm via HBASE-7842, but I'm not sure if
> one of would fix the other.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8408) Implement namespace
[
https://issues.apache.org/jira/browse/HBASE-8408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681634#comment-13681634
]
Hadoop QA commented on HBASE-8408:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585641/TestNamespaceUpgrade.tgz
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6023//console
This message is automatically generated.
> Implement namespace
> ---
>
> Key: HBASE-8408
> URL: https://issues.apache.org/jira/browse/HBASE-8408
> Project: HBase
> Issue Type: Sub-task
>Reporter: Francis Liu
>Assignee: Francis Liu
> Attachments: HBASE-8015_1.patch, HBASE-8015_2.patch,
> HBASE-8015_3.patch, HBASE-8015_4.patch, HBASE-8015_5.patch,
> HBASE-8015_6.patch, HBASE-8015.patch, TestNamespaceMigration.tgz,
> TestNamespaceUpgrade.tgz
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8702) Make WALEditCodec pluggable
[
https://issues.apache.org/jira/browse/HBASE-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681633#comment-13681633
]
Hadoop QA commented on HBASE-8702:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12587504/hbase-8702-0.94-v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6022//console
This message is automatically generated.
> Make WALEditCodec pluggable
> ---
>
> Key: HBASE-8702
> URL: https://issues.apache.org/jira/browse/HBASE-8702
> Project: HBase
> Issue Type: Improvement
> Components: Replication, wal
>Reporter: Jesse Yates
>Assignee: Jesse Yates
> Fix For: 0.98.0, 0.95.2, 0.94.9
>
> Attachments: hbase-8702-0.94-v0.patch, hbase-8702-0.94-v1.patch,
> hbase-8702-trunk-v0.patch, hbase-8702-trunk-v1.patch,
> hbase-8702-trunk-v2.patch
>
>
> WALEditCode needs to be pluggable to support alternative serialziation
> mechanisms.
> The open question here is whether to support the alternative codec when doing
> replication - both clusters would need the codec on the classpath, which has
> additional overhead and also will be a little bit complicated when making the
> WAL serialization backwards compatible in 0.94.
> This is the follow-up to HBASE-8636.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8737) [replication] Change replication RPC to use cell blocks
Chris Trezzo created HBASE-8737: --- Summary: [replication] Change replication RPC to use cell blocks Key: HBASE-8737 URL: https://issues.apache.org/jira/browse/HBASE-8737 Project: HBase Issue Type: Improvement Components: Replication Reporter: Chris Trezzo Assignee: Chris Trezzo Currently, the replication rpc that ships edits simply dumps the byte value of WAL edit key/value pairs into a protobuf message. Modify the replication rpc mechanism to use cell blocks so it can leverage encoding and compression. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8408) Implement namespace
[ https://issues.apache.org/jira/browse/HBASE-8408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-8408: --- Status: Patch Available (was: Open) review board link: https://reviews.apache.org/r/10167/diff/#index_header > Implement namespace > --- > > Key: HBASE-8408 > URL: https://issues.apache.org/jira/browse/HBASE-8408 > Project: HBase > Issue Type: Sub-task >Reporter: Francis Liu >Assignee: Francis Liu > Attachments: HBASE-8015_1.patch, HBASE-8015_2.patch, > HBASE-8015_3.patch, HBASE-8015_4.patch, HBASE-8015_5.patch, > HBASE-8015_6.patch, HBASE-8015.patch, TestNamespaceMigration.tgz, > TestNamespaceUpgrade.tgz > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4811) Support reverse Scan
[ https://issues.apache.org/jira/browse/HBASE-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681613#comment-13681613 ] Lars Hofhansl commented on HBASE-4811: -- bq. Sure, we could. Should we need rename "NonLazyKeyValueScanner"? Or only add new annotate for this class? I'd just add the new signatures. bq. backwardSeek is not only called when setting up I was trying to say that: # ReversedKeyValueHeap should not need backwardSeek, since all calls to (re)seek, etc, call backwardSeek anyway. # Only MemstoreScanner and StoreFileScanner should need an explicit backwardSeek method. Is that not so? (as I said I probably missed something). For a reversed scan we should get the following hierarchy: ReversedRegionScanner -> ReversedKVHeap -> ReversedStoreScanner -> ReversedKVHeap -> (MemstoreScanner|StoreFileScanner). Right? ReversedStoreScanner could always do the right (backward) seeking when (re)seek, etc, is called. The tricky parts of MemstoreScanner and StoreFileScanner since they are create/cached per memstore/HFile. So would: ReversedRegionScanner -> KVHelp -> RerversedStoreScanner -> ReversedKVHeap -> (MemstoreScanner|StoreFileScanner) work? The ReversedKVHeap would only be needed to deal with the specialness of MemstoreScanner and StoreFileScanner. > Support reverse Scan > > > Key: HBASE-4811 > URL: https://issues.apache.org/jira/browse/HBASE-4811 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.20.6, 0.94.7 >Reporter: John Carrino >Assignee: Liang Xie > Attachments: 4811-trunk-v10.txt, 4811-trunk-v5.patch, > HBase-4811-0.94.3modified.txt, HBase-4811-0.94-v2.txt, > hbase-4811-trunkv1.patch, hbase-4811-trunkv4.patch, hbase-4811-trunkv6.patch, > hbase-4811-trunkv7.patch, hbase-4811-trunkv8.patch, hbase-4811-trunkv9.patch > > > All the documentation I find about HBase says that if you want forward and > reverse scans you should just build 2 tables and one be ascending and one > descending. Is there a fundamental reason that HBase only supports forward > Scan? It seems like a lot of extra space overhead and coding overhead (to > keep them in sync) to support 2 tables. > I am assuming this has been discussed before, but I can't find the > discussions anywhere about it or why it would be infeasible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8702) Make WALEditCodec pluggable
[ https://issues.apache.org/jira/browse/HBASE-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-8702: --- Attachment: hbase-8702-0.94-v1.patch Updated patch for 0.94. Not significantly different from the other one, just a little nit cleanup. I didn't end up using ReflectionUtils since we don't have the HBase version in 0.94 and the stuff from Hadoop looks a bit too heavy weight - sooo its back to a nullary constructor (which you get by default anyways) and an empty init method. In the end, this ends up being just as easy for users to implement a WALEditCodec. I'd like to commit tonight (assuming the tests for this pass locally). > Make WALEditCodec pluggable > --- > > Key: HBASE-8702 > URL: https://issues.apache.org/jira/browse/HBASE-8702 > Project: HBase > Issue Type: Improvement > Components: Replication, wal >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.98.0, 0.95.2, 0.94.9 > > Attachments: hbase-8702-0.94-v0.patch, hbase-8702-0.94-v1.patch, > hbase-8702-trunk-v0.patch, hbase-8702-trunk-v1.patch, > hbase-8702-trunk-v2.patch > > > WALEditCode needs to be pluggable to support alternative serialziation > mechanisms. > The open question here is whether to support the alternative codec when doing > replication - both clusters would need the codec on the classpath, which has > additional overhead and also will be a little bit complicated when making the > WAL serialization backwards compatible in 0.94. > This is the follow-up to HBASE-8636. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8736) Clean up naming of constants for metrics
Elliott Clark created HBASE-8736: Summary: Clean up naming of constants for metrics Key: HBASE-8736 URL: https://issues.apache.org/jira/browse/HBASE-8736 Project: HBase Issue Type: Bug Components: metrics Affects Versions: 0.95.1, 0.98.0 Reporter: Elliott Clark Some metrics have their static names end with _NAME; others are missing that. All of them should end with _NAME to make it easy to find with a grep. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8702) Make WALEditCodec pluggable
[
https://issues.apache.org/jira/browse/HBASE-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681589#comment-13681589
]
Hadoop QA commented on HBASE-8702:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12587500/hbase-8702-trunk-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6021//console
This message is automatically generated.
> Make WALEditCodec pluggable
> ---
>
> Key: HBASE-8702
> URL: https://issues.apache.org/jira/browse/HBASE-8702
> Project: HBase
> Issue Type: Improvement
> Components: Replication, wal
>Reporter: Jesse Yates
>Assignee: Jesse Yates
> Fix For: 0.98.0, 0.95.2, 0.94.9
>
> Attachments: hbase-8702-0.94-v0.patch, hbase-8702-trunk-v0.patch,
> hbase-8702-trunk-v1.patch, hbase-8702-trunk-v2.patch
>
>
> WALEditCode needs to be pluggable to support alternative serialziation
> mechanisms.
> The open question here is whether to support the alternative codec when doing
> replication - both clusters would need the codec on the classpath, which has
> additional overhead and also will be a little bit complicated when making the
> WAL serialization backwards compatible in 0.94.
> This is the follow-up to HBASE-8636.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8702) Make WALEditCodec pluggable
[ https://issues.apache.org/jira/browse/HBASE-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-8702: --- Attachment: hbase-8702-trunk-v2.patch Attaching patch actually committed to trunk - fixes a javadoc warning and adding a final specifier to the CompressionContext in WALCellCodec (unnecessarily removed in last patch). > Make WALEditCodec pluggable > --- > > Key: HBASE-8702 > URL: https://issues.apache.org/jira/browse/HBASE-8702 > Project: HBase > Issue Type: Improvement > Components: Replication, wal >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.98.0, 0.95.2, 0.94.9 > > Attachments: hbase-8702-0.94-v0.patch, hbase-8702-trunk-v0.patch, > hbase-8702-trunk-v1.patch, hbase-8702-trunk-v2.patch > > > WALEditCode needs to be pluggable to support alternative serialziation > mechanisms. > The open question here is whether to support the alternative codec when doing > replication - both clusters would need the codec on the classpath, which has > additional overhead and also will be a little bit complicated when making the > WAL serialization backwards compatible in 0.94. > This is the follow-up to HBASE-8636. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8706) Some improvement in snapshot
[
https://issues.apache.org/jira/browse/HBASE-8706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-8706:
---
Resolution: Fixed
Fix Version/s: 0.95.2
0.98.0
Assignee: Matteo Bertozzi
Status: Resolved (was: Patch Available)
> Some improvement in snapshot
>
>
> Key: HBASE-8706
> URL: https://issues.apache.org/jira/browse/HBASE-8706
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.94.8, 0.95.0
>Reporter: binlijin
>Assignee: Matteo Bertozzi
> Fix For: 0.98.0, 0.95.2
>
> Attachments: HBASE-8706-2.patch, HBASE-8706-3.patch,
> HBASE-8706.patch, HBASE-8706-v4.patch, HBASE-8706-v4.patch
>
>
> (1)timeout for Procedure can not be configured.
> {code}
> Procedure's timeout
> ProcedureCoordinator
> final static long TIMEOUT_MILLIS_DEFAULT = 6;
>createProcedure(ForeignExceptionDispatcher fed, String procName, byte[]
> procArgs,
> List expectedMembers) {
> // build the procedure
> return new Procedure(this, fed, WAKE_MILLIS_DEFAULT,
> TIMEOUT_MILLIS_DEFAULT,
> procName, procArgs, expectedMembers);
> }
> RegionServerSnapshotManager:
> /** Conf key for max time to keep threads in snapshot request pool waiting
> */
> public static final String SNAPSHOT_TIMEOUT_MILLIS_KEY =
> "hbase.snapshot.region.timeout";
> /** Keep threads alive in request pool for max of 60 seconds */
> public static final long SNAPSHOT_TIMEOUT_MILLIS_DEFAULT = 6;
> public Subprocedure buildSubprocedure(SnapshotDescription snapshot) {
> long timeoutMillis = conf.getLong(SNAPSHOT_TIMEOUT_MILLIS_KEY,
> SNAPSHOT_TIMEOUT_MILLIS_DEFAULT);
> case FLUSH:
> SnapshotSubprocedurePool taskManager =
> new SnapshotSubprocedurePool(rss.getServerName().toString(), conf);
> }
> {code}
> (2)TakeSnapshotHandler
> after snapshotRegions we should call monitor.rethrowException(); to check if
> there is exception and if there is we can skip the verifySnapshot
> (3)too much error message when error happened in some place.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-7987) Snapshot Manifest file instead of multiple empty files
[
https://issues.apache.org/jira/browse/HBASE-7987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi reassigned HBASE-7987:
--
Assignee: Matteo Bertozzi
> Snapshot Manifest file instead of multiple empty files
> --
>
> Key: HBASE-7987
> URL: https://issues.apache.org/jira/browse/HBASE-7987
> Project: HBase
> Issue Type: Improvement
> Components: snapshots
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Attachments: HBASE-7987.sketch, HBASE-7987-v2.sketch
>
>
> Currently taking a snapshot means creating one empty file for each file in
> the source table directory, plus copying the .regioninfo file for each
> region, the table descriptor file and a snapshotInfo file.
> during the restore or snapshot verification we traverse the filesystem
> (fs.listStatus()) to find the snapshot files, and we open the .regioninfo
> files to get the information.
> to avoid hammering the NameNode and having lots of empty files, we can use a
> manifest file that contains the list of files and information that we need.
> To keep the RS parallelism that we have, each RS can write its own manifest.
> {code}
> message SnapshotDescriptor {
> required string name;
> optional string table;
> optional int64 creationTime;
> optional Type type;
> optional int32 version;
> }
> message SnapshotRegionManifest {
> optional int32 version;
> required RegionInfo regionInfo;
> repeated FamilyFiles familyFiles;
> message StoreFile {
> required string name;
> optional Reference reference;
> }
> message FamilyFiles {
> required bytes familyName;
> repeated StoreFile storeFiles;
> }
> }
> {code}
> {code}
> /hbase/.snapshot/
> /hbase/.snapshot//snapshotInfo
> /hbase/.snapshot//
> /hbase/.snapshot///tableInfo
> /hbase/.snapshot///regionManifest(.n)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8705) RS holding META when restarted in a single node setup may hang infinitely without META assignment
[
https://issues.apache.org/jira/browse/HBASE-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681397#comment-13681397
]
Jimmy Xiang commented on HBASE-8705:
+1, just one nit, should we reset the interruption flag in case an interrupted
exception?
> RS holding META when restarted in a single node setup may hang infinitely
> without META assignment
> -
>
> Key: HBASE-8705
> URL: https://issues.apache.org/jira/browse/HBASE-8705
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.1
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 0.98.0
>
> Attachments: HBASE-8705.patch
>
>
> This bug may be minor as it likely to happen in a single node setup.
> I restarted the RS holding META. The master tried assigning META using
> MetaSSH. But tried this before the new RS came up.
> So as not region plan is found
> {code}
> if (plan == null) {
> LOG.warn("Unable to determine a plan to assign " + region);
> if (tomActivated){
> this.timeoutMonitor.setAllRegionServersOffline(true);
> } else {
> regionStates.updateRegionState(region,
> RegionState.State.FAILED_OPEN);
> }
> return;
> }
> {code}
> we just return without assigment. And this being the META the small cluster
> just hangs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8541) implement flush-into-stripes in stripe compactions
[
https://issues.apache.org/jira/browse/HBASE-8541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681390#comment-13681390
]
Ted Yu commented on HBASE-8541:
---
{code}
+ public boolean doFlushToL0() {
+return flushIntoL0;
{code}
doFlushToL0 sounds like the method would do the actual flushing. Please rename.
{code}
- private class CompactionResultsMergeCopy {
+ private class CompactionOrFlushMergeCopy {
{code}
Mind updating javadoc for the above method w.r.t. flushing ?
{code}
+ if (isFlush) {
+long newSize =
StripeCompactionPolicy.getTotalFileSize(newStripes.values());
+LOG.error("Stripes were created by a flush, but results of size "
+ newSize
++ " cannot be added due to a concurrent flush also creating
stripes");
+canAddNewStripes = false;
+filesForL0 = newStripes.values();
{code}
In code above, newStripes is put in filesForL0 - meaning we ignore
conflictingFiles. But the error message says that newSize cannot be added. This
seems to be inconsistent, right ?
{code}
+public class StripeStoreFlusher extends StoreFlusher {
+ private static final Log LOG = LogFactory.getLog(DefaultStoreFlusher.class);
{code}
Add audience annotation.
Log gets wrong class.
{code}
+} finally {
+ if (!success && (mw != null)) {
+result.clear();
+for (StoreFile.Writer writer : mw.getAllWriters()) {
+ if (writer == null) continue;
+ writer.close();
{code}
You may want to catch IOException w.r.t. writer.close() call.
{code}
+if (scanner == null) {
+ return null; // NULL from coprocessor here means skip normal processing
+}
+return scanner;
{code}
'return scanner' should be good enough for above.
{code}
+ * @param targetCount The maximum number of stripes to flush into.
+ * @param targetKvs The KV count of each segment. If targetKvs*targetCount
is less than
{code}
Can the above two parameters get better names ? How about targetStripeCount and
targetKvsPerStripe ?
> implement flush-into-stripes in stripe compactions
> --
>
> Key: HBASE-8541
> URL: https://issues.apache.org/jira/browse/HBASE-8541
> Project: HBase
> Issue Type: Improvement
>Reporter: Sergey Shelukhin
> Attachments: HBASE-8541-latest-with-dependencies.patch,
> HBASE-8541-v0.patch
>
>
> Flush will be able to flush into multiple files under this design, avoiding
> L0 I/O amplification.
> I have the patch which is missing just one feature - support for concurrent
> flushes and stripe changes. This can be done via extensive try-locking of
> stripe changes and flushes, or advisory flags without blocking flushes,
> dumping conflicting flushes into L0 in case of (very rare) collisions. For
> file loading for the latter, a set-cover-like problem needs to be solved to
> determine optimal stripes. That will also address Jimmy's concern of getting
> rid of metadata, btw. However currently I don't have time for that. I plan to
> attach the try-locking patch first, but this won't happen for a couple weeks
> probably and should not block main reviews. Hopefully this will be added on
> top of main reviews.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8735) "backup" hbase masters should eventually become active if the "primary" never comes up.
Jonathan Hsieh created HBASE-8735: - Summary: "backup" hbase masters should eventually become active if the "primary" never comes up. Key: HBASE-8735 URL: https://issues.apache.org/jira/browse/HBASE-8735 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.95.1 Reporter: Jonathan Hsieh I was taking a look at my rig running hbase 0.95.1rc1 and it go to a point where a "backup" master process was stuck waiting for a primary master that was never going to show up. One master was down, and the other master was up but thinking it was only supposed to be a backup and stuck in this loop [1]. There was no master znode in zk, and the designated master was never going to succeed at starting (due to other reasons). I've was killing active master and using then using the start-all.sh script to start any killed process back up again -- my guess is that the backup-masters special case maybe getting the backup stuck in that loop -- and that since the "primary" never shows up it gets stuck there. In this case, I think the "backup" master should eventually take over instead of being stuck in the loop and having an hbase with no master. [1] https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L502L508) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8734) HBase side support for BIGTOP-1007 (Introduce a modules system for HBase coprocessor applications)
Andrew Purtell created HBASE-8734:
-
Summary: HBase side support for BIGTOP-1007 (Introduce a modules
system for HBase coprocessor applications)
Key: HBASE-8734
URL: https://issues.apache.org/jira/browse/HBASE-8734
Project: HBase
Issue Type: Improvement
Components: Coprocessors
Reporter: Andrew Purtell
Assignee: Andrew Purtell
BIGTOP-1007 proposes a modules system convention ("/etc/hbase/modules.d"), a
common pattern used for example by Apache httpd, for easily installation and
removal of HBase coprocessor applications.
Will propose any HBase side changes needed here, or close soon if none are
required.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8720) Only one snapshot region tasks that can run at a time
[
https://issues.apache.org/jira/browse/HBASE-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681357#comment-13681357
]
Matteo Bertozzi commented on HBASE-8720:
The current snapshot implementation is just a first cut, so some names and code
are there ready for the future. (Example, this concurrent name, the
restoreWALs() in the restore code and so on)
again, I don't think that having N idle thread around for the snapshot is a
good idea, changing the queue type can be a more acceptable patch, but still.
If you don't have a test and a benchmark that demonstrate that you have a real
performance gain having more thread doesn't make sense at this time.
You may consider introducing a a new snapshot procedure that doesn't flush the
memstore and and keeps track of the logs, and with that you may consider
increasing the number of threads, since is more likely to have a perf gain and
less downside.
> Only one snapshot region tasks that can run at a time
> -
>
> Key: HBASE-8720
> URL: https://issues.apache.org/jira/browse/HBASE-8720
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.94.8, 0.95.0
>Reporter: binlijin
> Attachments: 8720-v2.txt, HBASE-8720.patch
>
>
> {code}
> SnapshotSubprocedurePool(String name, Configuration conf) {
> // configure the executor service
> long keepAlive = conf.getLong(
> RegionServerSnapshotManager.SNAPSHOT_TIMEOUT_MILLIS_KEY,
> RegionServerSnapshotManager.SNAPSHOT_TIMEOUT_MILLIS_DEFAULT);
> int threads = conf.getInt(CONCURENT_SNAPSHOT_TASKS_KEY,
> DEFAULT_CONCURRENT_SNAPSHOT_TASKS);
> this.name = name;
> executor = new ThreadPoolExecutor(1, threads, keepAlive,
> TimeUnit.MILLISECONDS,
> new LinkedBlockingQueue(), new DaemonThreadFactory("rs("
> + name + ")-snapshot-pool"));
> taskPool = new ExecutorCompletionService(executor);
> }
> {code}
> ThreadPoolExecutor：
> corePoolSize：1
> maximumPoolSize：3
> workQueue：LinkedBlockingQueue，unlimited
> so when a new task submit to the ThreadPoolExecutor, if there is a task is
> running, the new task is queued in the queue, so all snapshot region tasks
> execute one by one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8721) Deletes can mask puts that happen after the delete
[ https://issues.apache.org/jira/browse/HBASE-8721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681338#comment-13681338 ] Andrew Purtell commented on HBASE-8721: --- Not debating the merits at this time. I did change this to be an improvement, since this proposes changing behavior that the dev community has consensus on being acceptable (Feng disagrees), with a configuration toggle, if I read the above discussion correctly. > Deletes can mask puts that happen after the delete > -- > > Key: HBASE-8721 > URL: https://issues.apache.org/jira/browse/HBASE-8721 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Feng Honghua > Attachments: HBASE-8721-0.94-V0.patch > > > this fix aims for bug mentioned in http://hbase.apache.org/book.html 5.8.2.1: > "Deletes mask puts, even puts that happened after the delete was entered. > Remember that a delete writes a tombstone, which only disappears after then > next major compaction has run. Suppose you do a delete of everything <= T. > After this you do a new put with a timestamp <= T. This put, even if it > happened after the delete, will be masked by the delete tombstone. Performing > the put will not fail, but when you do a get you will notice the put did have > no effect. It will start working again after the major compaction has run. > These issues should not be a problem if you use always-increasing versions > for new puts to a row. But they can occur even if you do not care about time: > just do delete and put immediately after each other, and there is some chance > they happen within the same millisecond." -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8720) Only one snapshot region tasks that can run at a time
[
https://issues.apache.org/jira/browse/HBASE-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681337#comment-13681337
]
binlijin commented on HBASE-8720:
-
hi Matteo Bertozzi ：
Queuing:
Any BlockingQueue may be used to transfer and hold submitted tasks. The use of
this queue interacts with pool sizing:
If fewer than corePoolSize threads are running, the Executor always prefers
adding a new thread rather than queuing.
If corePoolSize or more threads are running, the Executor always prefers
queuing a request rather than adding a new thread.
If a request cannot be queued, a new thread is created unless this would exceed
maximumPoolSize, in which case, the task will be rejected.
so when a new task submit to the ThreadPoolExecutor, if there is a task is
running, the new task is queued in the queue, so all snapshot region tasks
execute one by one.
And i think CONCURENT_SNAPSHOT_TASKS_KEY makes people misunderstanding.
/** Maximum number of snapshot region tasks that can run concurrently */
private static final String CONCURENT_SNAPSHOT_TASKS_KEY =
"hbase.snapshot.region.concurrentTasks";
private static final int DEFAULT_CONCURRENT_SNAPSHOT_TASKS = 3;
> Only one snapshot region tasks that can run at a time
> -
>
> Key: HBASE-8720
> URL: https://issues.apache.org/jira/browse/HBASE-8720
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.94.8, 0.95.0
>Reporter: binlijin
> Attachments: 8720-v2.txt, HBASE-8720.patch
>
>
> {code}
> SnapshotSubprocedurePool(String name, Configuration conf) {
> // configure the executor service
> long keepAlive = conf.getLong(
> RegionServerSnapshotManager.SNAPSHOT_TIMEOUT_MILLIS_KEY,
> RegionServerSnapshotManager.SNAPSHOT_TIMEOUT_MILLIS_DEFAULT);
> int threads = conf.getInt(CONCURENT_SNAPSHOT_TASKS_KEY,
> DEFAULT_CONCURRENT_SNAPSHOT_TASKS);
> this.name = name;
> executor = new ThreadPoolExecutor(1, threads, keepAlive,
> TimeUnit.MILLISECONDS,
> new LinkedBlockingQueue(), new DaemonThreadFactory("rs("
> + name + ")-snapshot-pool"));
> taskPool = new ExecutorCompletionService(executor);
> }
> {code}
> ThreadPoolExecutor：
> corePoolSize：1
> maximumPoolSize：3
> workQueue：LinkedBlockingQueue，unlimited
> so when a new task submit to the ThreadPoolExecutor, if there is a task is
> running, the new task is queued in the queue, so all snapshot region tasks
> execute one by one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8721) Deletes can mask puts that happen after the delete
[ https://issues.apache.org/jira/browse/HBASE-8721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-8721: -- Issue Type: Improvement (was: Bug) Summary: Deletes can mask puts that happen after the delete (was: fix for bug that delete can mask puts that happened after the delete was entered) > Deletes can mask puts that happen after the delete > -- > > Key: HBASE-8721 > URL: https://issues.apache.org/jira/browse/HBASE-8721 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Feng Honghua > Attachments: HBASE-8721-0.94-V0.patch > > > this fix aims for bug mentioned in http://hbase.apache.org/book.html 5.8.2.1: > "Deletes mask puts, even puts that happened after the delete was entered. > Remember that a delete writes a tombstone, which only disappears after then > next major compaction has run. Suppose you do a delete of everything <= T. > After this you do a new put with a timestamp <= T. This put, even if it > happened after the delete, will be masked by the delete tombstone. Performing > the put will not fail, but when you do a get you will notice the put did have > no effect. It will start working again after the major compaction has run. > These issues should not be a problem if you use always-increasing versions > for new puts to a row. But they can occur even if you do not care about time: > just do delete and put immediately after each other, and there is some chance > they happen within the same millisecond." -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-8721) fix for bug that delete can mask puts that happened after the delete was entered
[ https://issues.apache.org/jira/browse/HBASE-8721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reopened HBASE-8721: --- Reopening. There's discussion continuing and a reasonable defense of the proposal. > fix for bug that delete can mask puts that happened after the delete was > entered > > > Key: HBASE-8721 > URL: https://issues.apache.org/jira/browse/HBASE-8721 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Feng Honghua > Attachments: HBASE-8721-0.94-V0.patch > > > this fix aims for bug mentioned in http://hbase.apache.org/book.html 5.8.2.1: > "Deletes mask puts, even puts that happened after the delete was entered. > Remember that a delete writes a tombstone, which only disappears after then > next major compaction has run. Suppose you do a delete of everything <= T. > After this you do a new put with a timestamp <= T. This put, even if it > happened after the delete, will be masked by the delete tombstone. Performing > the put will not fail, but when you do a get you will notice the put did have > no effect. It will start working again after the major compaction has run. > These issues should not be a problem if you use always-increasing versions > for new puts to a row. But they can occur even if you do not care about time: > just do delete and put immediately after each other, and there is some chance > they happen within the same millisecond." -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8716) Fixups/Improvements for graceful_stop.sh/region_mover.rb
[ https://issues.apache.org/jira/browse/HBASE-8716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681327#comment-13681327 ] stack commented on HBASE-8716: -- Will commit this small change later today unless objection. > Fixups/Improvements for graceful_stop.sh/region_mover.rb > > > Key: HBASE-8716 > URL: https://issues.apache.org/jira/browse/HBASE-8716 > Project: HBase > Issue Type: Improvement >Reporter: stack >Assignee: stack > Fix For: 0.95.2 > > Attachments: 8716.txt > > > It is a while since these scripts were touched. Giving them a spring > cleaning and seeing if can make them return error codes on failure (seems > like style previous was that the operator would watch the output and react to > it but I see cases where tools want to call these scripts and they want > return code to indicate whether the rolling upgrade worked or not). Also, > see if can make the rolling restart faster since one-by-one while minimally > disruptive and 'safe', it is slow one clusters of hundreds of nodes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8720) Only one snapshot region tasks that can run at a time
[
https://issues.apache.org/jira/browse/HBASE-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681306#comment-13681306
]
Matteo Bertozzi commented on HBASE-8720:
I'm -1 on this patch...
The first argument of the ThreadPoolExecutor is used to defined how many idle
threads you want to keep around... and you want this number to be low
expecially for snapshots, since they are used just used in the order of 1 or 2
times per hour (I guess is around that if you're a heavy user)...
{code}
corePoolSize - the number of threads to keep in the pool, even if they are idle.
maximumPoolSize - the maximum number of threads to allow in the pool.
{code}
NOTE that increasing that number doesn't give you the parallelism, you have to
increase the number in the configuration property... and if you have space
(maxPoolSize) a new thread will be added.
Also the allowCoreThreadTimeOut(true) added in v2 is basically the same as
saying corePoolSize = 0, since you are asking to remove idle threads based on
the keepAlive.
another note is that having more threads may make things worse in the current
case. Each thread will trigger a flush, to take a snapshot, and this means more
I/O.. and you end up slowing down everything else.
If you want to spend more time on this patch
1. Add a test to demonstrate how many threads are really running during the
snapshot
2. Test and get performance result of what is the gain and the performance
degradation that you have by increasing the number of threads
> Only one snapshot region tasks that can run at a time
> -
>
> Key: HBASE-8720
> URL: https://issues.apache.org/jira/browse/HBASE-8720
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.94.8, 0.95.0
>Reporter: binlijin
> Attachments: 8720-v2.txt, HBASE-8720.patch
>
>
> {code}
> SnapshotSubprocedurePool(String name, Configuration conf) {
> // configure the executor service
> long keepAlive = conf.getLong(
> RegionServerSnapshotManager.SNAPSHOT_TIMEOUT_MILLIS_KEY,
> RegionServerSnapshotManager.SNAPSHOT_TIMEOUT_MILLIS_DEFAULT);
> int threads = conf.getInt(CONCURENT_SNAPSHOT_TASKS_KEY,
> DEFAULT_CONCURRENT_SNAPSHOT_TASKS);
> this.name = name;
> executor = new ThreadPoolExecutor(1, threads, keepAlive,
> TimeUnit.MILLISECONDS,
> new LinkedBlockingQueue(), new DaemonThreadFactory("rs("
> + name + ")-snapshot-pool"));
> taskPool = new ExecutorCompletionService(executor);
> }
> {code}
> ThreadPoolExecutor：
> corePoolSize：1
> maximumPoolSize：3
> workQueue：LinkedBlockingQueue，unlimited
> so when a new task submit to the ThreadPoolExecutor, if there is a task is
> running, the new task is queued in the queue, so all snapshot region tasks
> execute one by one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4360) Maintain information on the time a RS went dead
[ https://issues.apache.org/jira/browse/HBASE-4360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681235#comment-13681235 ] Nicolas Liochon commented on HBASE-4360: The DeadServer already contains a map for the death time. For consistency: - you should at the very least have exactly the same value. With the current patch we could have 1ms difference sometimes and this might create confusion in the future. - you should may be not add another field in ServerName(no strong opinion) - or you should change DeadServer to use the field you added in ServerName (no strong opinion) Now it's strange to have a class called ServerName that includes a death time. > Maintain information on the time a RS went dead > --- > > Key: HBASE-4360 > URL: https://issues.apache.org/jira/browse/HBASE-4360 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.94.0 >Reporter: Harsh J >Priority: Minor > Attachments: HBASE-4360_1.patch, HBASE-4360_2.patch, > master-status1.png > > > Just something that'd be generally helpful, is to maintain DeadServer info > with the last timestamp when it was determined as dead. > Makes it easier to hunt the logs, and I don't think its much too expensive to > maintain (one additional update per dead determination). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8733) Update our hadoop2 in trunk and 0.95 to 2.0.5-alpha (We are currently 2.0.2)
[ https://issues.apache.org/jira/browse/HBASE-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681220#comment-13681220 ] Nicolas Liochon commented on HBASE-8733: +1 > Update our hadoop2 in trunk and 0.95 to 2.0.5-alpha (We are currently 2.0.2) > > > Key: HBASE-8733 > URL: https://issues.apache.org/jira/browse/HBASE-8733 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.98.0, 0.95.2 > > Attachments: 205.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8700) IntegrationTestBigLinkedList can fail due to random number collision
[ https://issues.apache.org/jira/browse/HBASE-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681205#comment-13681205 ] Hudson commented on HBASE-8700: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #565 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/565/]) HBASE-8700 IntegrationTestBigLinkedList can fail due to random number collision (Revision 1492034) Result = FAILURE sershe : Files : * /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java > IntegrationTestBigLinkedList can fail due to random number collision > > > Key: HBASE-8700 > URL: https://issues.apache.org/jira/browse/HBASE-8700 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-8700-0.94.patch, HBASE-8700-v0.patch, > HBASE-8700-v1.patch > > > The test can fail due to random number collision, claiming there are > unreferenced elements for obvious reasons (we rewrite some link). Original > Accumulo test has one-stage generation so it doesn't count unreferenced > elements as failures, only undefined ones. With 200m longs out of half-long > range the probability of collision is approx 0.2%. > Moreover, without some way to debug, it's hard to debug what keys should be > looked at in such cases -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8721) fix for bug that delete can mask puts that happened after the delete was entered
[ https://issues.apache.org/jira/browse/HBASE-8721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681204#comment-13681204 ] Feng Honghua commented on HBASE-8721: - [~sershe] This fix is not about timestamp as a role to control version, but whether the behaviour 'delete can mask puts that happened after the delete" is reasonable/acceptable. I totally agree with you that timestamp is the only criterion to define/control version semantic, and this isn't broken by my fix. You can get what I mean by my above comment to Stack or by review and understand my patch. Thanks a lot > fix for bug that delete can mask puts that happened after the delete was > entered > > > Key: HBASE-8721 > URL: https://issues.apache.org/jira/browse/HBASE-8721 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Feng Honghua > Attachments: HBASE-8721-0.94-V0.patch > > > this fix aims for bug mentioned in http://hbase.apache.org/book.html 5.8.2.1: > "Deletes mask puts, even puts that happened after the delete was entered. > Remember that a delete writes a tombstone, which only disappears after then > next major compaction has run. Suppose you do a delete of everything <= T. > After this you do a new put with a timestamp <= T. This put, even if it > happened after the delete, will be masked by the delete tombstone. Performing > the put will not fail, but when you do a get you will notice the put did have > no effect. It will start working again after the major compaction has run. > These issues should not be a problem if you use always-increasing versions > for new puts to a row. But they can occur even if you do not care about time: > just do delete and put immediately after each other, and there is some chance > they happen within the same millisecond." -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8721) fix for bug that delete can mask puts that happened after the delete was entered
[ https://issues.apache.org/jira/browse/HBASE-8721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681200#comment-13681200 ] Feng Honghua commented on HBASE-8721: - [~stack] 1. I also agree keeping deleted cells forever(even during major compact) can fix the inconsistency in the scenario I mentioned before. But that treatment will make the HFile size continuously grow without shrinking by collecting deleted cells and delete flags. This is an obvious drawback by keeping deleted cells(together with delete flags) which is not desirable by many users. 2. The behaviour "delete can mask puts that happened after the delete" is unacceptable for many users. When a user puts a kv to HBase, his intention is to ADD that kv to HBase and definitely he want to be able to retrieve that kv back using a Get/Scan operation without regard to whether or not there is a delete ever occurred. Why current behaviour is unacceptable for two reasons: a> When a user puts a kv, receives success response, and fails to read it out, he'll be confused why and it's hard for him to realize that the reason is someone or himself ever wrote a delete before; b> If delete can mask puts happened after that delete, this means once a delete is written to HBase(till it's collected by major compact), it can block that kv be added back to HBase again forever(by semantic) even though that kv can be added back to HBase successfully using 'put' operation(by syntactic) 3. Yes, my fix is really to adjust the behaviour "delete can mask puts that happened after the delete" to the one that "delete can only mask puts that happened before(or equal) the delete". With this behaviour adjustment the inconsistency caused by major compact doesn't appear again. 4. My fix is using mvcc together with timestamp to determine whether a delete can mask a put. This treatment doesn't break the original delete semantic defined by timestamp alone, but enforced with mvcc that define the real ordering of time-point that operations occur(timestamp can't). Why I don't use sequence-id: a> when flushed/compacted to HFile, sequence-id doesn't accompany kv any longer, but mvcc does; if we use seq-id, we can't handle kv in hfile for this purpose; b> Yes the seq-id defines persistence ordering of kv and the mvcc defines visibility ordering of kv, and they can interleave for two kvs, but it doesn't hurt the correctness of our adjustment behaviour in that when seq-id advances user can't see(read) that kv until its mvcc advances. visibility occurs after persistence(mvcc is after seq-id). seq-id is background implementation detail and user isn't aware of it while mvcc impacts data visibility and user is aware of it. > fix for bug that delete can mask puts that happened after the delete was > entered > > > Key: HBASE-8721 > URL: https://issues.apache.org/jira/browse/HBASE-8721 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Feng Honghua > Attachments: HBASE-8721-0.94-V0.patch > > > this fix aims for bug mentioned in http://hbase.apache.org/book.html 5.8.2.1: > "Deletes mask puts, even puts that happened after the delete was entered. > Remember that a delete writes a tombstone, which only disappears after then > next major compaction has run. Suppose you do a delete of everything <= T. > After this you do a new put with a timestamp <= T. This put, even if it > happened after the delete, will be masked by the delete tombstone. Performing > the put will not fail, but when you do a get you will notice the put did have > no effect. It will start working again after the major compaction has run. > These issues should not be a problem if you use always-increasing versions > for new puts to a row. But they can occur even if you do not care about time: > just do delete and put immediately after each other, and there is some chance > they happen within the same millisecond." -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8700) IntegrationTestBigLinkedList can fail due to random number collision
[ https://issues.apache.org/jira/browse/HBASE-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681148#comment-13681148 ] Hudson commented on HBASE-8700: --- Integrated in hbase-0.95-on-hadoop2 #130 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/130/]) HBASE-8700 IntegrationTestBigLinkedList can fail due to random number collision (Revision 1492035) Result = FAILURE sershe : Files : * /hbase/branches/0.95/hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java > IntegrationTestBigLinkedList can fail due to random number collision > > > Key: HBASE-8700 > URL: https://issues.apache.org/jira/browse/HBASE-8700 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-8700-0.94.patch, HBASE-8700-v0.patch, > HBASE-8700-v1.patch > > > The test can fail due to random number collision, claiming there are > unreferenced elements for obvious reasons (we rewrite some link). Original > Accumulo test has one-stage generation so it doesn't count unreferenced > elements as failures, only undefined ones. With 200m longs out of half-long > range the probability of collision is approx 0.2%. > Moreover, without some way to debug, it's hard to debug what keys should be > looked at in such cases -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4811) Support reverse Scan
[ https://issues.apache.org/jira/browse/HBASE-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681132#comment-13681132 ] chunhui shen commented on HBASE-4811: - [~lhofhansl] Thanks for the review. bq.do we need NonReversedNonLazyKeyValueScanner? Could add "unsupported" implementations for these methods to NonLazyKeyValueScanner. Sure, we could. Should we need rename "NonLazyKeyValueScanner"? Or only add new annotate for this class? bq.should we have an initScan() (or maybe setup()) method on the scanners that does the right thing? I.e. a ReversedScanner would do the seekToLastRow/backwardSeek stuff, and a normal scanner would just seek. backwardSeek is not only called when setting up. For this point, I don't understand very much, it would be better if you could update the patch as you think. bq.It should either scan backwards or not? It is indeed weird calling backwardSeek in the method of reseek since reseek means forward seek. I would update the path with addressing the first and third point later > Support reverse Scan > > > Key: HBASE-4811 > URL: https://issues.apache.org/jira/browse/HBASE-4811 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.20.6, 0.94.7 >Reporter: John Carrino >Assignee: Liang Xie > Attachments: 4811-trunk-v10.txt, 4811-trunk-v5.patch, > HBase-4811-0.94.3modified.txt, HBase-4811-0.94-v2.txt, > hbase-4811-trunkv1.patch, hbase-4811-trunkv4.patch, hbase-4811-trunkv6.patch, > hbase-4811-trunkv7.patch, hbase-4811-trunkv8.patch, hbase-4811-trunkv9.patch > > > All the documentation I find about HBase says that if you want forward and > reverse scans you should just build 2 tables and one be ascending and one > descending. Is there a fundamental reason that HBase only supports forward > Scan? It seems like a lot of extra space overhead and coding overhead (to > keep them in sync) to support 2 tables. > I am assuming this has been discussed before, but I can't find the > discussions anywhere about it or why it would be infeasible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4360) Maintain information on the time a RS went dead
[ https://issues.apache.org/jira/browse/HBASE-4360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] samar updated HBASE-4360: - Status: Patch Available (was: Open) > Maintain information on the time a RS went dead > --- > > Key: HBASE-4360 > URL: https://issues.apache.org/jira/browse/HBASE-4360 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.94.0 >Reporter: Harsh J >Priority: Minor > Attachments: HBASE-4360_1.patch, HBASE-4360_2.patch, > master-status1.png > > > Just something that'd be generally helpful, is to maintain DeadServer info > with the last timestamp when it was determined as dead. > Makes it easier to hunt the logs, and I don't think its much too expensive to > maintain (one additional update per dead determination). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4360) Maintain information on the time a RS went dead
[ https://issues.apache.org/jira/browse/HBASE-4360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] samar updated HBASE-4360: - Attachment: HBASE-4360_2.patch with modified testcase > Maintain information on the time a RS went dead > --- > > Key: HBASE-4360 > URL: https://issues.apache.org/jira/browse/HBASE-4360 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.94.0 >Reporter: Harsh J >Priority: Minor > Attachments: HBASE-4360_1.patch, HBASE-4360_2.patch, > master-status1.png > > > Just something that'd be generally helpful, is to maintain DeadServer info > with the last timestamp when it was determined as dead. > Makes it easier to hunt the logs, and I don't think its much too expensive to > maintain (one additional update per dead determination). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8720) Only one snapshot region tasks that can run at a time
[
https://issues.apache.org/jira/browse/HBASE-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681020#comment-13681020
]
binlijin commented on HBASE-8720:
-
+1, LGTM.
> Only one snapshot region tasks that can run at a time
> -
>
> Key: HBASE-8720
> URL: https://issues.apache.org/jira/browse/HBASE-8720
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.94.8, 0.95.0
>Reporter: binlijin
> Attachments: 8720-v2.txt, HBASE-8720.patch
>
>
> {code}
> SnapshotSubprocedurePool(String name, Configuration conf) {
> // configure the executor service
> long keepAlive = conf.getLong(
> RegionServerSnapshotManager.SNAPSHOT_TIMEOUT_MILLIS_KEY,
> RegionServerSnapshotManager.SNAPSHOT_TIMEOUT_MILLIS_DEFAULT);
> int threads = conf.getInt(CONCURENT_SNAPSHOT_TASKS_KEY,
> DEFAULT_CONCURRENT_SNAPSHOT_TASKS);
> this.name = name;
> executor = new ThreadPoolExecutor(1, threads, keepAlive,
> TimeUnit.MILLISECONDS,
> new LinkedBlockingQueue(), new DaemonThreadFactory("rs("
> + name + ")-snapshot-pool"));
> taskPool = new ExecutorCompletionService(executor);
> }
> {code}
> ThreadPoolExecutor：
> corePoolSize：1
> maximumPoolSize：3
> workQueue：LinkedBlockingQueue，unlimited
> so when a new task submit to the ThreadPoolExecutor, if there is a task is
> running, the new task is queued in the queue, so all snapshot region tasks
> execute one by one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
