[jira] [Commented] (HBASE-11339) HBase LOB
[ https://issues.apache.org/jira/browse/HBASE-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030285#comment-14030285 ] Jonathan Hsieh commented on HBASE-11339: Nice doc. I did a quick read and have some design level questions and concerns: The core problem we are trying to avoid is write amplification (writing the data in the hlog, then in flush and then over and over again with compactions). Does the proposed design write out LOBs to both the HLog and then later LOB files? As designed, it must write them to the log so that we preserve durability and consistency properties of a row. + good that this should just would work with replication - in the best case, the data is written at least twice -- once before the ack is sent to the client and then again on flush. Can we limit this to once? We could avoid extra writes by just writing to a separate LOB log/file. Was this considered? Is there any consideration of locality and performance? 5MB cells are large but aren't really that big. Maybe this should just be blobs (binary large objects) or mobs (medium objects)? the objects being immutable is important too. HBase LOB - Key: HBASE-11339 URL: https://issues.apache.org/jira/browse/HBASE-11339 Project: HBase Issue Type: New Feature Components: regionserver, Scanners Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: HBase LOB Design.pdf It's quite useful to save the massive binary data like images, documents into Apache HBase. Unfortunately directly saving the binary LOB(large object) to HBase leads to a worse performance since the frequent split and compaction. In this design, the LOB data are stored in an more efficient way, which keeps a high write/read performance and guarantees the data consistency in Apache HBase. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10926) Use global procedure to flush table memstore cache
[ https://issues.apache.org/jira/browse/HBASE-10926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030287#comment-14030287 ] Jerry He commented on HBASE-10926: -- The normal layout of the zk node for the 'reached' phase: : |-/hbase/flush-table-proc : |-reached : |cluster_test : |---hdtest010.svl.ibm.com,60020,1402634727563 : |---hdtest011.svl.ibm.com,60020,1402634727508 procName = ZKUtil.getNodeName(ZKUtil.getParent(path)) -- cluster_test member = ZKUtil.getNodeName(path)-- hdtest010.svl.ibm.com,60020,1402634727563 But from the warning message, we seem to have the wrong layout (my deduction, not actual test result): : |-/hbase/flush-table-proc : |-reached : |cluster_test The leaf nodes were missing. Therefore we got: procName = ZKUtil.getNodeName(ZKUtil.getParent(path)) -- reached member = ZKUtil.getNodeName(path) -- cluster_test Either the leaf node was not created or cleared due to race? Use global procedure to flush table memstore cache -- Key: HBASE-10926 URL: https://issues.apache.org/jira/browse/HBASE-10926 Project: HBase Issue Type: Improvement Components: Admin Affects Versions: 0.96.2, 0.98.1 Reporter: Jerry He Assignee: Jerry He Fix For: 0.99.0 Attachments: HBASE-10926-trunk-v1.patch, HBASE-10926-trunk-v2.patch, HBASE-10926-trunk-v3.patch, HBASE-10926-trunk-v4.patch Currently, user can trigger table flush through hbase shell or HBaseAdmin API. To flush the table cache, each region server hosting the regions is contacted and flushed sequentially, which is less efficient. In HBase snapshot global procedure is used to coordinate and flush the regions in a distributed way. Let's provide a distributed table flush for general use. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10926) Use global procedure to flush table memstore cache
[ https://issues.apache.org/jira/browse/HBASE-10926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030288#comment-14030288 ] Jerry He commented on HBASE-10926: -- BTW. I am on vacation in China :-) Access to gmail and google is slow :-( Use global procedure to flush table memstore cache -- Key: HBASE-10926 URL: https://issues.apache.org/jira/browse/HBASE-10926 Project: HBase Issue Type: Improvement Components: Admin Affects Versions: 0.96.2, 0.98.1 Reporter: Jerry He Assignee: Jerry He Fix For: 0.99.0 Attachments: HBASE-10926-trunk-v1.patch, HBASE-10926-trunk-v2.patch, HBASE-10926-trunk-v3.patch, HBASE-10926-trunk-v4.patch Currently, user can trigger table flush through hbase shell or HBaseAdmin API. To flush the table cache, each region server hosting the regions is contacted and flushed sequentially, which is less efficient. In HBase snapshot global procedure is used to coordinate and flush the regions in a distributed way. Let's provide a distributed table flush for general use. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11162) RegionServer webui uses the default master info port irrespective of the user configuration.
[ https://issues.apache.org/jira/browse/HBASE-11162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030299#comment-14030299 ] Hudson commented on HBASE-11162: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #315 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/315/]) HBASE-11162 RegionServer webui uses the default master info port irrespective of the user configuration (Srikanth) (jxiang: rev 3cd5cf81939ce6470544855f9d26423e27e0bc76) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java RegionServer webui uses the default master info port irrespective of the user configuration. Key: HBASE-11162 URL: https://issues.apache.org/jira/browse/HBASE-11162 Project: HBase Issue Type: Bug Components: UI Affects Versions: 0.98.2 Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Priority: Minor Fix For: 0.99.0, 0.98.4 Attachments: HBASE-11162.patch, HBASE-11162_trunk.patch Under the regionserver ui software attributes section, the value for attribute HBase Master is always using the default port (60010), even if the user configured master info port to a different value. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030323#comment-14030323
]
Hadoop QA commented on HBASE-4495:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12650221/HBASE-4495.patch
against trunk revision .
ATTACHMENT ID: 12650221
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 124
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 5
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ public static int getRegionCount(final Configuration c, final String
tableName) throws IOException {
+ * Adds a (single) hbase:meta row for the specified new region and its
daughters. Note that this does
+ * Adds a (single) hbase:meta row for the specified new region and its
daughters. Note that this does
+ final HRegionInfo regionInfo, final
ServerName sn, final long openSeqNum)
+ HRegionInfo mergedRegion, HRegionInfo
regionA, HRegionInfo regionB,
+ private static void multiMutate(HTable table, byte[] row, Mutation...
mutations) throws IOException {
+MultiRowMutationProtos.MutateRowsRequest.Builder mmrBuilder =
MultiRowMutationProtos.MutateRowsRequest.newBuilder();
+
mmrBuilder.addMutationRequest(ProtobufUtil.toMutation(ClientProtos.MutationProto.MutationType.PUT,
mutation));
+
mmrBuilder.addMutationRequest(ProtobufUtil.toMutation(ClientProtos.MutationProto.MutationType.DELETE,
mutation));
+PairHRegionInfo, ServerName pair =
MetaTableAccessor.getRegion(connection, tableNameOrRegionName);
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.catalog.TestMetaRegionLocator.testInterruptWaitOnMeta(TestMetaRegionLocator.java:163)
at
org.apache.hadoop.hbase.master.TestAssignmentManager.testSSHTimesOutOpeningRegionTransition(TestAssignmentManager.java:1050)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9762//console
This message is automatically generated.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Attachments: HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an
[jira] [Created] (HBASE-11340) Remove references to xcievers in documentation
Dima Spivak created HBASE-11340: --- Summary: Remove references to xcievers in documentation Key: HBASE-11340 URL: https://issues.apache.org/jira/browse/HBASE-11340 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Assignee: Dima Spivak Priority: Trivial While most references to xcievers were removed in HBase as part of [HBASE-5697], the ref guide still contains a few stray references. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11340) Remove references to xcievers in documentation
[ https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11340: Status: Patch Available (was: Open) Remove references to xcievers in documentation -- Key: HBASE-11340 URL: https://issues.apache.org/jira/browse/HBASE-11340 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Assignee: Dima Spivak Priority: Trivial Attachments: HBASE-11340.patch While most references to xcievers were removed in HBase as part of [HBASE-5697], the ref guide still contains a few stray references. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Work stopped] (HBASE-11340) Remove references to xcievers in documentation
[ https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-11340 stopped by Dima Spivak. Remove references to xcievers in documentation -- Key: HBASE-11340 URL: https://issues.apache.org/jira/browse/HBASE-11340 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Assignee: Dima Spivak Priority: Trivial Attachments: HBASE-11340.patch While most references to xcievers were removed in HBase as part of [HBASE-5697], the ref guide still contains a few stray references. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Work started] (HBASE-11340) Remove references to xcievers in documentation
[ https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-11340 started by Dima Spivak. Remove references to xcievers in documentation -- Key: HBASE-11340 URL: https://issues.apache.org/jira/browse/HBASE-11340 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Assignee: Dima Spivak Priority: Trivial Attachments: HBASE-11340.patch While most references to xcievers were removed in HBase as part of [HBASE-5697], the ref guide still contains a few stray references. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11340) Remove references to xcievers in documentation
[ https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11340: Attachment: HBASE-11340.patch Remove references to xcievers in documentation -- Key: HBASE-11340 URL: https://issues.apache.org/jira/browse/HBASE-11340 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Assignee: Dima Spivak Priority: Trivial Attachments: HBASE-11340.patch While most references to xcievers were removed in HBase as part of [HBASE-5697], the ref guide still contains a few stray references. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10885) Support visibility expressions on Deletes
[
https://issues.apache.org/jira/browse/HBASE-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030338#comment-14030338
]
ramkrishna.s.vasudevan commented on HBASE-10885:
bq.then we need to check if the effective auths of the user who issued the
delete grant visibility to the covered cell. Only if true do we delete it.
Oh.. Then this becomes more complex. So not only the cell visibility
associated with the delete needs to be considered, in cases that is not present
then go with the Auths defined in the labels table? In that sense I feel
passing Cell visibility itself is not needed right? Apply deletes only based
on the Auths associated with the user issuing the delete. Also if this
behaviour we need not only we need to match the exact labels but also do as in
Scan and find out if the labels in the cell is a subset of the auths and then
decide the delete behaviour.
bq.#2 is the semantic change from earlier versions.
I agree that we can change the semantics without which we are not allowing the
deletes to function properly.
I was testing more scenarios today and ended up in more tricky situations
{code}
put 't1','r1','f1:c1','100',101,{VISIBILITY='PRIVATE|SECRET'}
put 't1','r1','f1:c1','100',102,{VISIBILITY='PRIVATE'}
put
't1','r1','f1:c1','100',103,{VISIBILITY='(SECRETTOPSECRET)|(PRIVATECONFIDENTIAL)'}
put 't1','r1','f1:c1','100',104,{VISIBILITY='(SECRETTOPSECRET)'}
put
't1','r1','f1:c1','100',105,{VISIBILITY='(SECRETTOPSECRET)|(PRIVATECONFIDENTIAL)'}
{code}
Now issue these two deletes
{code}
delete_column_timeStamp 't1','r1','f1:c1',104,'SECRETTOPSECRET'
delete_familyversion
't1','r1','f1:c1',103,'(CONFIDENTIALPRIVATE)|(TOPSECRETSECRET)'
{code}
Now when we do
{code}
delete_family 't1','r1','f1','PRIVATE|SECRET'
{code}
Then we are bound to get back all the previously deleted cells also as per the
current patch because DELETE_FAMILY being a higher precedence we don check for
lower types of deletes and only match the visibility labels for those cells
with PRIVATE|SECRET and this makes already deleted cells to reappear.
Support visibility expressions on Deletes
-
Key: HBASE-10885
URL: https://issues.apache.org/jira/browse/HBASE-10885
Project: HBase
Issue Type: Improvement
Affects Versions: 0.98.1
Reporter: Andrew Purtell
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
Fix For: 0.99.0, 0.98.4
Attachments: HBASE-10885_1.patch, HBASE-10885_2.patch,
HBASE-10885_new_tag_type_1.patch, HBASE-10885_new_tag_type_2.patch,
HBASE-10885_v1.patch, HBASE-10885_v2.patch, HBASE-10885_v2.patch,
HBASE-10885_v2.patch, HBASE-10885_v3.patch
Accumulo can specify visibility expressions for delete markers. During
compaction the cells covered by the tombstone are determined in part by
matching the visibility expression. This is useful for the use case of data
set coalescing, where entries from multiple data sets carrying different
labels are combined into one common large table. Later, a subset of entries
can be conveniently removed using visibility expressions.
Currently doing the same in HBase would only be possible with a custom
coprocessor. Otherwise, a Delete will affect all cells covered by the
tombstone regardless of any visibility expression scoping. This is correct
behavior in that no data spill is possible, but certainly could be
surprising, and is only meant to be transitional. We decided not to support
visibility expressions on Deletes to control the complexity of the initial
implementation.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10926) Use global procedure to flush table memstore cache
[ https://issues.apache.org/jira/browse/HBASE-10926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030341#comment-14030341 ] Matteo Bertozzi commented on HBASE-10926: - ZKProcedureCoordinatorRpcs receives a nodeCreated() event on /hbase/flush-table-proc/reached/cluster_test and when we check isReachedPathNode() we don't verify if the node is the member or the root. so the warn is caused by the root creation event. Use global procedure to flush table memstore cache -- Key: HBASE-10926 URL: https://issues.apache.org/jira/browse/HBASE-10926 Project: HBase Issue Type: Improvement Components: Admin Affects Versions: 0.96.2, 0.98.1 Reporter: Jerry He Assignee: Jerry He Fix For: 0.99.0 Attachments: HBASE-10926-trunk-v1.patch, HBASE-10926-trunk-v2.patch, HBASE-10926-trunk-v3.patch, HBASE-10926-trunk-v4.patch Currently, user can trigger table flush through hbase shell or HBaseAdmin API. To flush the table cache, each region server hosting the regions is contacted and flushed sequentially, which is less efficient. In HBase snapshot global procedure is used to coordinate and flush the regions in a distributed way. Let's provide a distributed table flush for general use. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11338) Expand documentation on bloom filters
[ https://issues.apache.org/jira/browse/HBASE-11338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11338: Component/s: (was: Filters) Expand documentation on bloom filters - Key: HBASE-11338 URL: https://issues.apache.org/jira/browse/HBASE-11338 Project: HBase Issue Type: Bug Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Ref Guide could use more info on bloom filters. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11107) Provide utility method equivalent to 0.92's Result.getBytes().getSize()
[
https://issues.apache.org/jira/browse/HBASE-11107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030358#comment-14030358
]
Hudson commented on HBASE-11107:
FAILURE: Integrated in HBase-TRUNK #5200 (See
[https://builds.apache.org/job/HBase-TRUNK/5200/])
HBASE-11107 Provide utility method equivalent to 0.92's
Result.getBytes().getSize() (Gustavo Anatoly) (tedyu: rev
62a78ab661ef432dfe054d0c15258057250fbc78)
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java
Provide utility method equivalent to 0.92's Result.getBytes().getSize()
---
Key: HBASE-11107
URL: https://issues.apache.org/jira/browse/HBASE-11107
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Gustavo Anatoly
Priority: Trivial
Fix For: 0.99.0
Attachments: HBASE-11107.patch
Currently user has to write code similar to the following for replacement of
Result.getBytes().getSize() :
{code}
+Cell[] cellValues = resultRow.rawCells();
+
+long size = 0L;
+if (null != cellValues) {
+ for (Cell cellValue : cellValues) {
+size += KeyValueUtil.ensureKeyValue(cellValue).heapSize();
+ }
+}
{code}
In ClientScanner, we have:
{code}
for (Cell kv : rs.rawCells()) {
// TODO make method in Cell or CellUtil
remainingResultSize -=
KeyValueUtil.ensureKeyValue(kv).heapSize();
}
{code}
A utility method should be provided which computes summation of Cell sizes in
a Result.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11094) Distributed log replay is incompatible for rolling restarts
[ https://issues.apache.org/jira/browse/HBASE-11094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030360#comment-14030360 ] Hudson commented on HBASE-11094: FAILURE: Integrated in HBase-TRUNK #5200 (See [https://builds.apache.org/job/HBase-TRUNK/5200/]) HBase-11094: Distributed log replay is incompatible for rolling restarts (jeffreyz: rev 55cecc9679ca04796450d75e72cfc89e1f3b4970) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterFileSystem.java * hbase-server/src/main/java/org/apache/hadoop/hbase/SplitLogTask.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java * hbase-protocol/src/main/protobuf/Admin.proto * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogMethods.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java * hbase-protocol/src/main/protobuf/ZooKeeper.proto * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/HLogSplitterHandler.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitLogWorker.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerNoMaster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java Distributed log replay is incompatible for rolling restarts --- Key: HBASE-11094 URL: https://issues.apache.org/jira/browse/HBASE-11094 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Jeffrey Zhong Priority: Blocker Fix For: 0.99.0 Attachments: hbase-11094-v2.patch, hbase-11094-v3.patch, hbase-11094-v4.patch, hbase-11094-v5.1.patch, hbase-11094-v5.patch, hbase-11094.patch 0.99.0 comes with dist log replay by default (HBASE-10888). However, reading the code and discussing this with Jeffrey, we realized that the dist log replay code is not compatible with rolling upgrades from 0.98.0 and 1.0.0. The issue is that, the region server looks at it own configuration to decide whether the region should be opened in replay mode or not. The open region RPC does not contain that info. So if dist log replay is enabled on master, the master will assign the region and schedule replay tasks. If the region is opened in a RS that does not have this conf enabled, then it will happily open the region in normal mode (not replay mode) causing possible (transient) data loss. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11162) RegionServer webui uses the default master info port irrespective of the user configuration.
[ https://issues.apache.org/jira/browse/HBASE-11162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030359#comment-14030359 ] Hudson commented on HBASE-11162: FAILURE: Integrated in HBase-TRUNK #5200 (See [https://builds.apache.org/job/HBase-TRUNK/5200/]) HBASE-11162 RegionServer webui uses the default master info port irrespective of the user configuration (Srikanth) (jxiang: rev 9e824cda192013d605d62d227b07ac00b76c3b32) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java RegionServer webui uses the default master info port irrespective of the user configuration. Key: HBASE-11162 URL: https://issues.apache.org/jira/browse/HBASE-11162 Project: HBase Issue Type: Bug Components: UI Affects Versions: 0.98.2 Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Priority: Minor Fix For: 0.99.0, 0.98.4 Attachments: HBASE-11162.patch, HBASE-11162_trunk.patch Under the regionserver ui software attributes section, the value for attribute HBase Master is always using the default port (60010), even if the user configured master info port to a different value. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11340) Remove references to xcievers in documentation
[
https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030371#comment-14030371
]
Hadoop QA commented on HBASE-11340:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12650237/HBASE-11340.patch
against trunk revision .
ATTACHMENT ID: 12650237
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testWritesWhileGetting(TestHRegion.java:3499)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9763//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9763//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9763//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9763//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9763//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9763//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9763//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9763//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9763//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9763//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9763//console
This message is automatically generated.
Remove references to xcievers in documentation
--
Key: HBASE-11340
URL: https://issues.apache.org/jira/browse/HBASE-11340
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Dima Spivak
Assignee: Dima Spivak
Priority: Trivial
Attachments: HBASE-11340.patch
While most references to xcievers were removed in HBase as part of
[HBASE-5697], the ref guide still contains a few stray references.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11341) ZKProcedureCoordinatorRpcs should respond only to members
Matteo Bertozzi created HBASE-11341:
---
Summary: ZKProcedureCoordinatorRpcs should respond only to members
Key: HBASE-11341
URL: https://issues.apache.org/jira/browse/HBASE-11341
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.3, 0.94.20, 0.99.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 0.99.0, 0.94.21, 0.98.4
ZKProcedureCoordinatorRpcs nodeCreated() responds to events of every znode
under the procedure and not only members.
{code}
INFO: Received created event: /hbase/proc/reached/instance
INFO: Received created event: /hbase/proc/reached/instance/member
{code}
the result is a warning (no other side effects)
{code}
WARN [main-EventThread] procedure.ProcedureCoordinator: Member 'cluster_test'
is trying to release an unknown procedure 'reached
{code}
[~apurtell] posted in HBASE-10926 the steps to reproduce it:
{noformat}
$ cd ./src/hbase
$ git checkout master
$ mvn -DskipTests clean install
{noformat}
In one console:
{noformat}
$ ./bin/hbase master start
{noformat}
In another console:
{noformat}
$ ./bin/hbase org.apache.hadoop.hbase.util.LoadTestTool -num_keys 10 \
-read 10:100 -write 1:100:10
{noformat}
In a third console:
{noformat}
$ ./bin/hbase shell
hbase while true do ; flush 'cluster_test' ; sleep 10 ; end
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11341) ZKProcedureCoordinatorRpcs should respond only to members
[
https://issues.apache.org/jira/browse/HBASE-11341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-11341:
Attachment: HBASE-11341-v0.patch
ZKProcedureCoordinatorRpcs should respond only to members
-
Key: HBASE-11341
URL: https://issues.apache.org/jira/browse/HBASE-11341
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0, 0.94.20, 0.98.3
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 0.99.0, 0.94.21, 0.98.4
Attachments: HBASE-11341-v0.patch
ZKProcedureCoordinatorRpcs nodeCreated() responds to events of every znode
under the procedure and not only members.
{code}
INFO: Received created event: /hbase/proc/reached/instance
INFO: Received created event: /hbase/proc/reached/instance/member
{code}
the result is a warning (no other side effects)
{code}
WARN [main-EventThread] procedure.ProcedureCoordinator: Member 'cluster_test'
is trying to release an unknown procedure 'reached
{code}
[~apurtell] posted in HBASE-10926 the steps to reproduce it:
{noformat}
$ cd ./src/hbase
$ git checkout master
$ mvn -DskipTests clean install
{noformat}
In one console:
{noformat}
$ ./bin/hbase master start
{noformat}
In another console:
{noformat}
$ ./bin/hbase org.apache.hadoop.hbase.util.LoadTestTool -num_keys 10 \
-read 10:100 -write 1:100:10
{noformat}
In a third console:
{noformat}
$ ./bin/hbase shell
hbase while true do ; flush 'cluster_test' ; sleep 10 ; end
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11341) ZKProcedureCoordinatorRpcs should respond only to members
[
https://issues.apache.org/jira/browse/HBASE-11341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-11341:
Description:
ZKProcedureCoordinatorRpcs nodeCreated() responds to events of every znode
under the procedure and not only members.
{code}
INFO: Received created event: /hbase/proc/reached/instance
INFO: Received created event: /hbase/proc/reached/instance/member
{code}
the result is a warning (no other side effects)
{code}
WARN [main-EventThread] procedure.ProcedureCoordinator: Member 'instance' is
trying to release an unknown procedure 'reached
{code}
[~apurtell] posted in HBASE-10926 the steps to reproduce it:
{noformat}
$ cd ./src/hbase
$ git checkout master
$ mvn -DskipTests clean install
{noformat}
In one console:
{noformat}
$ ./bin/hbase master start
{noformat}
In another console:
{noformat}
$ ./bin/hbase org.apache.hadoop.hbase.util.LoadTestTool -num_keys 10 \
-read 10:100 -write 1:100:10
{noformat}
In a third console:
{noformat}
$ ./bin/hbase shell
hbase while true do ; flush 'cluster_test' ; sleep 10 ; end
{noformat}
was:
ZKProcedureCoordinatorRpcs nodeCreated() responds to events of every znode
under the procedure and not only members.
{code}
INFO: Received created event: /hbase/proc/reached/instance
INFO: Received created event: /hbase/proc/reached/instance/member
{code}
the result is a warning (no other side effects)
{code}
WARN [main-EventThread] procedure.ProcedureCoordinator: Member 'cluster_test'
is trying to release an unknown procedure 'reached
{code}
[~apurtell] posted in HBASE-10926 the steps to reproduce it:
{noformat}
$ cd ./src/hbase
$ git checkout master
$ mvn -DskipTests clean install
{noformat}
In one console:
{noformat}
$ ./bin/hbase master start
{noformat}
In another console:
{noformat}
$ ./bin/hbase org.apache.hadoop.hbase.util.LoadTestTool -num_keys 10 \
-read 10:100 -write 1:100:10
{noformat}
In a third console:
{noformat}
$ ./bin/hbase shell
hbase while true do ; flush 'cluster_test' ; sleep 10 ; end
{noformat}
ZKProcedureCoordinatorRpcs should respond only to members
-
Key: HBASE-11341
URL: https://issues.apache.org/jira/browse/HBASE-11341
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0, 0.94.20, 0.98.3
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 0.99.0, 0.94.21, 0.98.4
Attachments: HBASE-11341-v0.patch
ZKProcedureCoordinatorRpcs nodeCreated() responds to events of every znode
under the procedure and not only members.
{code}
INFO: Received created event: /hbase/proc/reached/instance
INFO: Received created event: /hbase/proc/reached/instance/member
{code}
the result is a warning (no other side effects)
{code}
WARN [main-EventThread] procedure.ProcedureCoordinator: Member 'instance'
is trying to release an unknown procedure 'reached
{code}
[~apurtell] posted in HBASE-10926 the steps to reproduce it:
{noformat}
$ cd ./src/hbase
$ git checkout master
$ mvn -DskipTests clean install
{noformat}
In one console:
{noformat}
$ ./bin/hbase master start
{noformat}
In another console:
{noformat}
$ ./bin/hbase org.apache.hadoop.hbase.util.LoadTestTool -num_keys 10 \
-read 10:100 -write 1:100:10
{noformat}
In a third console:
{noformat}
$ ./bin/hbase shell
hbase while true do ; flush 'cluster_test' ; sleep 10 ; end
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11341) ZKProcedureCoordinatorRpcs should respond only to members
[
https://issues.apache.org/jira/browse/HBASE-11341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-11341:
Status: Patch Available (was: Open)
ZKProcedureCoordinatorRpcs should respond only to members
-
Key: HBASE-11341
URL: https://issues.apache.org/jira/browse/HBASE-11341
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.3, 0.94.20, 0.99.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 0.99.0, 0.94.21, 0.98.4
Attachments: HBASE-11341-v0.patch
ZKProcedureCoordinatorRpcs nodeCreated() responds to events of every znode
under the procedure and not only members.
{code}
INFO: Received created event: /hbase/proc/reached/instance
INFO: Received created event: /hbase/proc/reached/instance/member
{code}
the result is a warning (no other side effects)
{code}
WARN [main-EventThread] procedure.ProcedureCoordinator: Member 'instance'
is trying to release an unknown procedure 'reached
{code}
[~apurtell] posted in HBASE-10926 the steps to reproduce it:
{noformat}
$ cd ./src/hbase
$ git checkout master
$ mvn -DskipTests clean install
{noformat}
In one console:
{noformat}
$ ./bin/hbase master start
{noformat}
In another console:
{noformat}
$ ./bin/hbase org.apache.hadoop.hbase.util.LoadTestTool -num_keys 10 \
-read 10:100 -write 1:100:10
{noformat}
In a third console:
{noformat}
$ ./bin/hbase shell
hbase while true do ; flush 'cluster_test' ; sleep 10 ; end
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11342) The method isChildReadLock in class ZKInterProcessLockBase is wrong
Qianxi Zhang created HBASE-11342:
Summary: The method isChildReadLock in class
ZKInterProcessLockBase is wrong
Key: HBASE-11342
URL: https://issues.apache.org/jira/browse/HBASE-11342
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 0.98.3, 0.99.0
Reporter: Qianxi Zhang
Assignee: Qianxi Zhang
Priority: Minor
The method isChildReadLock in class ZKInterProcessLockBase may be wrong, which
determines whether the lock is readLock or not. So we should compare the node
name with READ_LOCK_CHILD_NODE_PREFIX rather than WRITE_LOCK_CHILD_NODE_PREFIX.
Since there is no other method to invoke the method isChildReadLock now, we
have not encountered an error.
{code}
protected static boolean isChildReadLock(String child) {
int idx = child.lastIndexOf(ZKUtil.ZNODE_PATH_SEPARATOR);
String suffix = child.substring(idx + 1);
return suffix.startsWith(WRITE_LOCK_CHILD_NODE_PREFIX);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11107) Provide utility method equivalent to 0.92's Result.getBytes().getSize()
[
https://issues.apache.org/jira/browse/HBASE-11107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030543#comment-14030543
]
Gustavo Anatoly commented on HBASE-11107:
-
Thank you, Ted.
Thank you, Chunhui
Provide utility method equivalent to 0.92's Result.getBytes().getSize()
---
Key: HBASE-11107
URL: https://issues.apache.org/jira/browse/HBASE-11107
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Gustavo Anatoly
Priority: Trivial
Fix For: 0.99.0
Attachments: HBASE-11107.patch
Currently user has to write code similar to the following for replacement of
Result.getBytes().getSize() :
{code}
+Cell[] cellValues = resultRow.rawCells();
+
+long size = 0L;
+if (null != cellValues) {
+ for (Cell cellValue : cellValues) {
+size += KeyValueUtil.ensureKeyValue(cellValue).heapSize();
+ }
+}
{code}
In ClientScanner, we have:
{code}
for (Cell kv : rs.rawCells()) {
// TODO make method in Cell or CellUtil
remainingResultSize -=
KeyValueUtil.ensureKeyValue(kv).heapSize();
}
{code}
A utility method should be provided which computes summation of Cell sizes in
a Result.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11342) The method isChildReadLock in class ZKInterProcessLockBase is wrong
[
https://issues.apache.org/jira/browse/HBASE-11342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qianxi Zhang updated HBASE-11342:
-
Attachment: HBASE_11342.patch
The method isChildReadLock in class ZKInterProcessLockBase is wrong
---
Key: HBASE-11342
URL: https://issues.apache.org/jira/browse/HBASE-11342
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 0.99.0, 0.98.3
Reporter: Qianxi Zhang
Assignee: Qianxi Zhang
Priority: Minor
Attachments: HBASE_11342.patch
The method isChildReadLock in class ZKInterProcessLockBase may be wrong,
which determines whether the lock is readLock or not. So we should compare
the node name with READ_LOCK_CHILD_NODE_PREFIX rather than
WRITE_LOCK_CHILD_NODE_PREFIX. Since there is no other method to invoke the
method isChildReadLock now, we have not encountered an error.
{code}
protected static boolean isChildReadLock(String child) {
int idx = child.lastIndexOf(ZKUtil.ZNODE_PATH_SEPARATOR);
String suffix = child.substring(idx + 1);
return suffix.startsWith(WRITE_LOCK_CHILD_NODE_PREFIX);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10926) Use global procedure to flush table memstore cache
[ https://issues.apache.org/jira/browse/HBASE-10926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030585#comment-14030585 ] Jerry He commented on HBASE-10926: -- Right, we don't verify. But on the other hand, we don't 'watch' for the parent 'root' path on the coordinator side, we only 'watch' for the expected member path. Right? Use global procedure to flush table memstore cache -- Key: HBASE-10926 URL: https://issues.apache.org/jira/browse/HBASE-10926 Project: HBase Issue Type: Improvement Components: Admin Affects Versions: 0.96.2, 0.98.1 Reporter: Jerry He Assignee: Jerry He Fix For: 0.99.0 Attachments: HBASE-10926-trunk-v1.patch, HBASE-10926-trunk-v2.patch, HBASE-10926-trunk-v3.patch, HBASE-10926-trunk-v4.patch Currently, user can trigger table flush through hbase shell or HBaseAdmin API. To flush the table cache, each region server hosting the regions is contacted and flushed sequentially, which is less efficient. In HBase snapshot global procedure is used to coordinate and flush the regions in a distributed way. Let's provide a distributed table flush for general use. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11331) [blockcache] lazy block decompression
[ https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030586#comment-14030586 ] Anoop Sam John commented on HBASE-11331: Not decrypting gives (so much) size save? bq.decompressing every time I read from a block rather than as we have now where we decompress once Ya a concerning factor and +1 for making it configurable. [blockcache] lazy block decompression - Key: HBASE-11331 URL: https://issues.apache.org/jira/browse/HBASE-11331 Project: HBase Issue Type: Improvement Components: regionserver Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-11331.00.patch Maintaining data in its compressed form in the block cache will greatly increase our effective blockcache size and should show a meaning improvement in cache hit rates in well designed applications. The idea here is to lazily decompress/decrypt blocks when they're consumed, rather than as soon as they're pulled off of disk. This is related to but less invasive than HBASE-8894. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10885) Support visibility expressions on Deletes
[ https://issues.apache.org/jira/browse/HBASE-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-10885: --- Status: Open (was: Patch Available) Support visibility expressions on Deletes - Key: HBASE-10885 URL: https://issues.apache.org/jira/browse/HBASE-10885 Project: HBase Issue Type: Improvement Affects Versions: 0.98.1 Reporter: Andrew Purtell Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.99.0, 0.98.4 Attachments: HBASE-10885_1.patch, HBASE-10885_2.patch, HBASE-10885_new_tag_type_1.patch, HBASE-10885_new_tag_type_2.patch, HBASE-10885_v1.patch, HBASE-10885_v2.patch, HBASE-10885_v2.patch, HBASE-10885_v2.patch, HBASE-10885_v3.patch Accumulo can specify visibility expressions for delete markers. During compaction the cells covered by the tombstone are determined in part by matching the visibility expression. This is useful for the use case of data set coalescing, where entries from multiple data sets carrying different labels are combined into one common large table. Later, a subset of entries can be conveniently removed using visibility expressions. Currently doing the same in HBase would only be possible with a custom coprocessor. Otherwise, a Delete will affect all cells covered by the tombstone regardless of any visibility expression scoping. This is correct behavior in that no data spill is possible, but certainly could be surprising, and is only meant to be transitional. We decided not to support visibility expressions on Deletes to control the complexity of the initial implementation. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10926) Use global procedure to flush table memstore cache
[
https://issues.apache.org/jira/browse/HBASE-10926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030588#comment-14030588
]
Matteo Bertozzi commented on HBASE-10926:
-
{quote}But on the other hand, we don't 'watch' for the parent 'root' path on
the coordinator side, we only 'watch' for the expected member path.
Right?{quote}
The watcher is on the ProcedureCoordinator not on the members.
we have a single watcher for all the acquired/reached/abort events.
so we get the parent events too
Use global procedure to flush table memstore cache
--
Key: HBASE-10926
URL: https://issues.apache.org/jira/browse/HBASE-10926
Project: HBase
Issue Type: Improvement
Components: Admin
Affects Versions: 0.96.2, 0.98.1
Reporter: Jerry He
Assignee: Jerry He
Fix For: 0.99.0
Attachments: HBASE-10926-trunk-v1.patch, HBASE-10926-trunk-v2.patch,
HBASE-10926-trunk-v3.patch, HBASE-10926-trunk-v4.patch
Currently, user can trigger table flush through hbase shell or HBaseAdmin
API. To flush the table cache, each region server hosting the regions is
contacted and flushed sequentially, which is less efficient.
In HBase snapshot global procedure is used to coordinate and flush the
regions in a distributed way.
Let's provide a distributed table flush for general use.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10885) Support visibility expressions on Deletes
[ https://issues.apache.org/jira/browse/HBASE-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-10885: --- Status: Patch Available (was: Open) Support visibility expressions on Deletes - Key: HBASE-10885 URL: https://issues.apache.org/jira/browse/HBASE-10885 Project: HBase Issue Type: Improvement Affects Versions: 0.98.1 Reporter: Andrew Purtell Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.99.0, 0.98.4 Attachments: HBASE-10885_1.patch, HBASE-10885_2.patch, HBASE-10885_new_tag_type_1.patch, HBASE-10885_new_tag_type_2.patch, HBASE-10885_v1.patch, HBASE-10885_v2.patch, HBASE-10885_v2.patch, HBASE-10885_v2.patch, HBASE-10885_v3.patch, HBASE-10885_v4.patch Accumulo can specify visibility expressions for delete markers. During compaction the cells covered by the tombstone are determined in part by matching the visibility expression. This is useful for the use case of data set coalescing, where entries from multiple data sets carrying different labels are combined into one common large table. Later, a subset of entries can be conveniently removed using visibility expressions. Currently doing the same in HBase would only be possible with a custom coprocessor. Otherwise, a Delete will affect all cells covered by the tombstone regardless of any visibility expression scoping. This is correct behavior in that no data spill is possible, but certainly could be surprising, and is only meant to be transitional. We decided not to support visibility expressions on Deletes to control the complexity of the initial implementation. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10885) Support visibility expressions on Deletes
[ https://issues.apache.org/jira/browse/HBASE-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-10885: --- Attachment: HBASE-10885_v4.patch Latest patch updating the comments. The delete family related issue is solved. The delete cells with and without Visibility tags is not yet handled. I think we could discuss and then proceed on that. Support visibility expressions on Deletes - Key: HBASE-10885 URL: https://issues.apache.org/jira/browse/HBASE-10885 Project: HBase Issue Type: Improvement Affects Versions: 0.98.1 Reporter: Andrew Purtell Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.99.0, 0.98.4 Attachments: HBASE-10885_1.patch, HBASE-10885_2.patch, HBASE-10885_new_tag_type_1.patch, HBASE-10885_new_tag_type_2.patch, HBASE-10885_v1.patch, HBASE-10885_v2.patch, HBASE-10885_v2.patch, HBASE-10885_v2.patch, HBASE-10885_v3.patch, HBASE-10885_v4.patch Accumulo can specify visibility expressions for delete markers. During compaction the cells covered by the tombstone are determined in part by matching the visibility expression. This is useful for the use case of data set coalescing, where entries from multiple data sets carrying different labels are combined into one common large table. Later, a subset of entries can be conveniently removed using visibility expressions. Currently doing the same in HBase would only be possible with a custom coprocessor. Otherwise, a Delete will affect all cells covered by the tombstone regardless of any visibility expression scoping. This is correct behavior in that no data spill is possible, but certainly could be surprising, and is only meant to be transitional. We decided not to support visibility expressions on Deletes to control the complexity of the initial implementation. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10933) hbck -fixHdfsOrphans is not working properly it throws null pointer exception
[
https://issues.apache.org/jira/browse/HBASE-10933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kashif J S updated HBASE-10933:
---
Attachment: HBASE-10933-trunk-v2.patch
Upload trunk patch with proper formatting
hbck -fixHdfsOrphans is not working properly it throws null pointer exception
-
Key: HBASE-10933
URL: https://issues.apache.org/jira/browse/HBASE-10933
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.94.16, 0.98.2
Reporter: Deepak Sharma
Assignee: Kashif J S
Priority: Critical
Fix For: 0.99.0, 0.94.21
Attachments: HBASE-10933-0.94-v1.patch, HBASE-10933-0.94-v2.patch,
HBASE-10933-trunk-v1.patch, HBASE-10933-trunk-v2.patch, TestResults-0.94.txt,
TestResults-trunk.txt
if we regioninfo file is not existing in hbase region then if we run hbck
repair or hbck -fixHdfsOrphans
then it is not able to resolve this problem it throws null pointer exception
{code}
2014-04-08 20:11:49,750 INFO [main] util.HBaseFsck
(HBaseFsck.java:adoptHdfsOrphans(470)) - Attempting to handle orphan hdfs
dir:
hdfs://10.18.40.28:54310/hbase/TestHdfsOrphans1/5a3de9ca65e587cb05c9384a3981c950
java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.HBaseFsck$TableInfo.access$000(HBaseFsck.java:1939)
at
org.apache.hadoop.hbase.util.HBaseFsck.adoptHdfsOrphan(HBaseFsck.java:497)
at
org.apache.hadoop.hbase.util.HBaseFsck.adoptHdfsOrphans(HBaseFsck.java:471)
at
org.apache.hadoop.hbase.util.HBaseFsck.restoreHdfsIntegrity(HBaseFsck.java:591)
at
org.apache.hadoop.hbase.util.HBaseFsck.offlineHdfsIntegrityRepair(HBaseFsck.java:369)
at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:447)
at org.apache.hadoop.hbase.util.HBaseFsck.exec(HBaseFsck.java:3769)
at org.apache.hadoop.hbase.util.HBaseFsck.run(HBaseFsck.java:3587)
at
com.huawei.isap.test.smartump.hadoop.hbase.HbaseHbckRepair.repairToFixHdfsOrphans(HbaseHbckRepair.java:244)
at
com.huawei.isap.test.smartump.hadoop.hbase.HbaseHbckRepair.setUp(HbaseHbckRepair.java:84)
at junit.framework.TestCase.runBare(TestCase.java:132)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
{code}
problem i got it is because since in HbaseFsck class
{code}
private void adoptHdfsOrphan(HbckInfo hi)
{code}
we are intializing tableinfo using SortedMapString, TableInfo tablesInfo
object
{code}
TableInfo tableInfo = tablesInfo.get(tableName);
{code}
but in private SortedMapString, TableInfo loadHdfsRegionInfos()
{code}
for (HbckInfo hbi: hbckInfos) {
if (hbi.getHdfsHRI() == null) {
// was an orphan
continue;
}
{code}
we have check if a region is orphan then that table will can not be added in
SortedMapString, TableInfo tablesInfo
so later while using this we get null pointer exception
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11341) ZKProcedureCoordinatorRpcs should respond only to members
[
https://issues.apache.org/jira/browse/HBASE-11341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030599#comment-14030599
]
Hadoop QA commented on HBASE-11341:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12650265/HBASE-11341-v0.patch
against trunk revision .
ATTACHMENT ID: 12650265
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9764//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9764//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9764//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9764//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9764//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9764//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9764//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9764//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9764//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9764//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9764//console
This message is automatically generated.
ZKProcedureCoordinatorRpcs should respond only to members
-
Key: HBASE-11341
URL: https://issues.apache.org/jira/browse/HBASE-11341
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0, 0.94.20, 0.98.3
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 0.99.0, 0.94.21, 0.98.4
Attachments: HBASE-11341-v0.patch
ZKProcedureCoordinatorRpcs nodeCreated() responds to events of every znode
under the procedure and not only members.
{code}
INFO: Received created event: /hbase/proc/reached/instance
INFO: Received created event: /hbase/proc/reached/instance/member
{code}
the result is a warning (no other side effects)
{code}
WARN [main-EventThread] procedure.ProcedureCoordinator: Member 'instance'
is trying to release an unknown procedure 'reached
{code}
[~apurtell] posted in HBASE-10926 the steps to reproduce it:
{noformat}
$ cd ./src/hbase
$ git checkout master
$ mvn -DskipTests clean install
{noformat}
In one console:
{noformat}
$ ./bin/hbase master start
{noformat}
In another console:
{noformat}
$ ./bin/hbase org.apache.hadoop.hbase.util.LoadTestTool -num_keys 10 \
-read 10:100 -write 1:100:10
{noformat}
In a third console:
{noformat}
$ ./bin/hbase shell
hbase while true do ; flush 'cluster_test' ; sleep 10 ; end
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11088) Support Visibility Expression Deletes in Shell
[ https://issues.apache.org/jira/browse/HBASE-11088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11088: --- Attachment: HBASE-11088_3.patch Support Visibility Expression Deletes in Shell -- Key: HBASE-11088 URL: https://issues.apache.org/jira/browse/HBASE-11088 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.99.0, 0.98.4 Attachments: HBASE-11058.patch, HBASE-11058_2.patch, HBASE-11088_3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11088) Support Visibility Expression Deletes in Shell
[ https://issues.apache.org/jira/browse/HBASE-11088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11088: --- Status: Patch Available (was: Open) Support Visibility Expression Deletes in Shell -- Key: HBASE-11088 URL: https://issues.apache.org/jira/browse/HBASE-11088 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.99.0, 0.98.4 Attachments: HBASE-11058.patch, HBASE-11058_2.patch, HBASE-11088_3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10926) Use global procedure to flush table memstore cache
[ https://issues.apache.org/jira/browse/HBASE-10926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030635#comment-14030635 ] Jerry He commented on HBASE-10926: -- On the region server side, we watch for '/hbase/flush-table-proc/reached/cluster_test'. The master and region server are the same JVM? and share the same ZK watcher? Use global procedure to flush table memstore cache -- Key: HBASE-10926 URL: https://issues.apache.org/jira/browse/HBASE-10926 Project: HBase Issue Type: Improvement Components: Admin Affects Versions: 0.96.2, 0.98.1 Reporter: Jerry He Assignee: Jerry He Fix For: 0.99.0 Attachments: HBASE-10926-trunk-v1.patch, HBASE-10926-trunk-v2.patch, HBASE-10926-trunk-v3.patch, HBASE-10926-trunk-v4.patch Currently, user can trigger table flush through hbase shell or HBaseAdmin API. To flush the table cache, each region server hosting the regions is contacted and flushed sequentially, which is less efficient. In HBase snapshot global procedure is used to coordinate and flush the regions in a distributed way. Let's provide a distributed table flush for general use. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11088) Support Visibility Expression Deletes in Shell
[
https://issues.apache.org/jira/browse/HBASE-11088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030683#comment-14030683
]
Hadoop QA commented on HBASE-11088:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12650296/HBASE-11088_3.patch
against trunk revision .
ATTACHMENT ID: 12650296
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+
#
+
#
+
#
+
#
+# Deletes all versions of the specified column with a timestamp less than
or equal to the specified timestamp.
+
#
+
#
+# Deletes all versions of the specified family with less than or equal to
the specified timestamp
+
#
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9765//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9765//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9765//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9765//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9765//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9765//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9765//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9765//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9765//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9765//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9765//console
This message is automatically generated.
Support Visibility Expression Deletes in Shell
--
Key: HBASE-11088
URL: https://issues.apache.org/jira/browse/HBASE-11088
Project: HBase
Issue Type: Sub-task
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
Fix For: 0.99.0, 0.98.4
Attachments: HBASE-11058.patch, HBASE-11058_2.patch,
HBASE-11088_3.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11341) ZKProcedureCoordinatorRpcs should respond only to members
[
https://issues.apache.org/jira/browse/HBASE-11341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030697#comment-14030697
]
stack commented on HBASE-11341:
---
+1
ZKProcedureCoordinatorRpcs should respond only to members
-
Key: HBASE-11341
URL: https://issues.apache.org/jira/browse/HBASE-11341
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0, 0.94.20, 0.98.3
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 0.99.0, 0.94.21, 0.98.4
Attachments: HBASE-11341-v0.patch
ZKProcedureCoordinatorRpcs nodeCreated() responds to events of every znode
under the procedure and not only members.
{code}
INFO: Received created event: /hbase/proc/reached/instance
INFO: Received created event: /hbase/proc/reached/instance/member
{code}
the result is a warning (no other side effects)
{code}
WARN [main-EventThread] procedure.ProcedureCoordinator: Member 'instance'
is trying to release an unknown procedure 'reached
{code}
[~apurtell] posted in HBASE-10926 the steps to reproduce it:
{noformat}
$ cd ./src/hbase
$ git checkout master
$ mvn -DskipTests clean install
{noformat}
In one console:
{noformat}
$ ./bin/hbase master start
{noformat}
In another console:
{noformat}
$ ./bin/hbase org.apache.hadoop.hbase.util.LoadTestTool -num_keys 10 \
-read 10:100 -write 1:100:10
{noformat}
In a third console:
{noformat}
$ ./bin/hbase shell
hbase while true do ; flush 'cluster_test' ; sleep 10 ; end
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10933) hbck -fixHdfsOrphans is not working properly it throws null pointer exception
[
https://issues.apache.org/jira/browse/HBASE-10933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030685#comment-14030685
]
Hadoop QA commented on HBASE-10933:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12650294/HBASE-10933-trunk-v2.patch
against trunk revision .
ATTACHMENT ID: 12650294
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9767//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9767//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9767//console
This message is automatically generated.
hbck -fixHdfsOrphans is not working properly it throws null pointer exception
-
Key: HBASE-10933
URL: https://issues.apache.org/jira/browse/HBASE-10933
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.94.16, 0.98.2
Reporter: Deepak Sharma
Assignee: Kashif J S
Priority: Critical
Fix For: 0.99.0, 0.94.21
Attachments: HBASE-10933-0.94-v1.patch, HBASE-10933-0.94-v2.patch,
HBASE-10933-trunk-v1.patch, HBASE-10933-trunk-v2.patch, TestResults-0.94.txt,
TestResults-trunk.txt
if we regioninfo file is not existing in hbase region then if we run hbck
repair or hbck -fixHdfsOrphans
then it is not able to resolve this problem it throws null pointer exception
{code}
2014-04-08 20:11:49,750 INFO [main] util.HBaseFsck
(HBaseFsck.java:adoptHdfsOrphans(470)) - Attempting to handle orphan hdfs
dir:
hdfs://10.18.40.28:54310/hbase/TestHdfsOrphans1/5a3de9ca65e587cb05c9384a3981c950
java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.HBaseFsck$TableInfo.access$000(HBaseFsck.java:1939)
at
org.apache.hadoop.hbase.util.HBaseFsck.adoptHdfsOrphan(HBaseFsck.java:497)
at
org.apache.hadoop.hbase.util.HBaseFsck.adoptHdfsOrphans(HBaseFsck.java:471)
at
org.apache.hadoop.hbase.util.HBaseFsck.restoreHdfsIntegrity(HBaseFsck.java:591)
at
org.apache.hadoop.hbase.util.HBaseFsck.offlineHdfsIntegrityRepair(HBaseFsck.java:369)
at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:447)
at org.apache.hadoop.hbase.util.HBaseFsck.exec(HBaseFsck.java:3769)
at org.apache.hadoop.hbase.util.HBaseFsck.run(HBaseFsck.java:3587)
at
com.huawei.isap.test.smartump.hadoop.hbase.HbaseHbckRepair.repairToFixHdfsOrphans(HbaseHbckRepair.java:244)
at
com.huawei.isap.test.smartump.hadoop.hbase.HbaseHbckRepair.setUp(HbaseHbckRepair.java:84)
at
[jira] [Commented] (HBASE-10885) Support visibility expressions on Deletes
[
https://issues.apache.org/jira/browse/HBASE-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030705#comment-14030705
]
Hadoop QA commented on HBASE-10885:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12650293/HBASE-10885_v4.patch
against trunk revision .
ATTACHMENT ID: 12650293
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestMultiParallel
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testWritesWhileGetting(TestHRegion.java:3499)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9766//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9766//console
This message is automatically generated.
Support visibility expressions on Deletes
-
Key: HBASE-10885
URL: https://issues.apache.org/jira/browse/HBASE-10885
Project: HBase
Issue Type: Improvement
Affects Versions: 0.98.1
Reporter: Andrew Purtell
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
Fix For: 0.99.0, 0.98.4
Attachments: HBASE-10885_1.patch, HBASE-10885_2.patch,
HBASE-10885_new_tag_type_1.patch, HBASE-10885_new_tag_type_2.patch,
HBASE-10885_v1.patch, HBASE-10885_v2.patch, HBASE-10885_v2.patch,
HBASE-10885_v2.patch, HBASE-10885_v3.patch, HBASE-10885_v4.patch
Accumulo can specify visibility expressions for delete markers. During
compaction the cells covered by the tombstone are determined in part by
matching the visibility expression. This is useful for the use case of data
set coalescing, where entries from multiple data sets carrying different
labels are combined into one common large table. Later, a subset of entries
can be conveniently removed using visibility expressions.
Currently doing the same in HBase would only be possible with a custom
coprocessor. Otherwise, a Delete will affect all cells covered by the
tombstone regardless of any visibility expression scoping. This is correct
behavior in that no data spill is possible, but certainly could be
surprising, and is only meant to be transitional. We decided not to support
visibility expressions on Deletes to control the complexity of the initial
implementation.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4920) We need a mascot, a totem
[ https://issues.apache.org/jira/browse/HBASE-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030717#comment-14030717 ] Nick Dimiduk commented on HBASE-4920: - My first choice is still https://issues.apache.org/jira/secure/attachment/12511412/HBase%20Orca%20Logo.jpg Second favorite is page 1 from https://issues.apache.org/jira/secure/attachment/12648671/Apache_HBase_Orca_Logo_round5.pdf We need a mascot, a totem - Key: HBASE-4920 URL: https://issues.apache.org/jira/browse/HBASE-4920 Project: HBase Issue Type: Task Reporter: stack Attachments: Apache_HBase_Orca_Logo_1.jpg, Apache_HBase_Orca_Logo_Mean_version-3.pdf, Apache_HBase_Orca_Logo_Mean_version-4.pdf, Apache_HBase_Orca_Logo_round5.pdf, HBase Orca Logo.jpg, Orca_479990801.jpg, Screen shot 2011-11-30 at 4.06.17 PM.png, apache hbase orca logo_Proof 3.pdf, apache logo_Proof 8.pdf, krake.zip, more_orcas.png, more_orcas2.png, photo (2).JPG, plus_orca.png We need a totem for our t-shirt that is yet to be printed. O'Reilly owns the Clyesdale. We need something else. We could have a fluffy little duck that quacks 'hbase!' when you squeeze it and we could order boxes of them from some off-shore sweatshop that subcontracts to a contractor who employs child labor only. Or we could have an Orca (Big!, Fast!, Killer!, and in a poem that Marcy from Salesforce showed me, that was a bit too spiritual for me to be seen quoting here, it had the Orca as the 'Guardian of the Cosmic Memory': i.e. in translation, bigdata). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-8844) Document the removal of replication state AKA start/stop_replication
[ https://issues.apache.org/jira/browse/HBASE-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-8844: -- Summary: Document the removal of replication state AKA start/stop_replication (was: Document stop_replication danger) Updating the jira's summary to better reflect what we ended up doing. Document the removal of replication state AKA start/stop_replication Key: HBASE-8844 URL: https://issues.apache.org/jira/browse/HBASE-8844 Project: HBase Issue Type: Task Components: documentation Reporter: Patrick Assignee: Misty Stanley-Jones Priority: Minor Attachments: HBASE-8844-v2.patch, HBASE-8844-v3-rebased.patch, HBASE-8844-v3.patch, HBASE-8844.patch The first two tutorials for enabling replication that google gives me [1], [2] take very different tones with regard to stop_replication. The HBase docs [1] make it sound fine to start and stop replication as desired. The Cloudera docs [2] say it may cause data loss. Which is true? If data loss is possible, are we talking about data loss in the primary cluster, or data loss in the standby cluster (presumably would require reinitializing the sync with a new CopyTable). [1] http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/replication/package-summary.html#requirements [2] http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/cdh4ig_topic_20_11.html -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11343) Writing the RootRegionLocation znode in thrift format.
Manukranth Kolloju created HBASE-11343: -- Summary: Writing the RootRegionLocation znode in thrift format. Key: HBASE-11343 URL: https://issues.apache.org/jira/browse/HBASE-11343 Project: HBase Issue Type: Improvement Affects Versions: 0.89-fb Reporter: Manukranth Kolloju Assignee: Manukranth Kolloju Priority: Minor Fix For: 0.89-fb -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-8844) Document the removal of replication state AKA start/stop_replication
[ https://issues.apache.org/jira/browse/HBASE-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-8844: -- Resolution: Fixed Fix Version/s: 0.99.0 Status: Resolved (was: Patch Available) Pushed to trunk, thanks Misty! Document the removal of replication state AKA start/stop_replication Key: HBASE-8844 URL: https://issues.apache.org/jira/browse/HBASE-8844 Project: HBase Issue Type: Task Components: documentation Reporter: Patrick Assignee: Misty Stanley-Jones Priority: Minor Fix For: 0.99.0 Attachments: HBASE-8844-v2.patch, HBASE-8844-v3-rebased.patch, HBASE-8844-v3.patch, HBASE-8844.patch The first two tutorials for enabling replication that google gives me [1], [2] take very different tones with regard to stop_replication. The HBase docs [1] make it sound fine to start and stop replication as desired. The Cloudera docs [2] say it may cause data loss. Which is true? If data loss is possible, are we talking about data loss in the primary cluster, or data loss in the standby cluster (presumably would require reinitializing the sync with a new CopyTable). [1] http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/replication/package-summary.html#requirements [2] http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/cdh4ig_topic_20_11.html -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-11056) Make multiget work more like batchMutate
[ https://issues.apache.org/jira/browse/HBASE-11056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-11056. -- Resolution: Duplicate Duplicate of HBASE-8362. Make multiget work more like batchMutate Key: HBASE-11056 URL: https://issues.apache.org/jira/browse/HBASE-11056 Project: HBase Issue Type: Improvement Components: regionserver Reporter: Nick Dimiduk Right now, the multiget path issues a sequence of Region.get calls for the single RPC. This is good, but it could be better. Multiple Puts are batched together and applied as a group, I think we can do the same for Gets. The idea is to reuse scanners for all Gets in the batch. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10871) Indefinite OPEN/CLOSE wait on busy RegionServers
[ https://issues.apache.org/jira/browse/HBASE-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-10871: Resolution: Fixed Fix Version/s: 0.98.4 0.94.21 0.99.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I updated the 0.94 patch a little and integrated it into 0.94. The trunk patch is integrated into trunk and 0.98 as-is. Thanks Esteban for the patch. Indefinite OPEN/CLOSE wait on busy RegionServers Key: HBASE-10871 URL: https://issues.apache.org/jira/browse/HBASE-10871 Project: HBase Issue Type: Improvement Components: Balancer, master, Region Assignment Affects Versions: 0.94.6 Reporter: Harsh J Assignee: Esteban Gutierrez Fix For: 0.99.0, 0.94.21, 0.98.4 Attachments: HBASE-10871-0.94.v1.patch, HBASE-10871.v0.patch, HBASE-10871.v1.patch We observed a case where, when a specific RS got bombarded by a large amount of regular requests, spiking and filling up its RPC queue, the balancer's invoked unassigns and assigns for regions that dealt with this server entered into an indefinite retry loop. The regions specifically began waiting in PENDING_CLOSE/PENDING_OPEN states indefinitely cause of the HBase Client RPC from the ServerManager at the master was running into SocketTimeouts. This caused a region unavailability in the server for the affected regions. The timeout monitor retry default of 30m in 0.94's AM compounded the waiting gap further a bit more (this is now 10m in 0.95+'s new AM, and has further retries before we get there, which is good). Wonder if there's a way to improve this situation generally. PENDING_OPENs may be easy to handle - we can switch them out and move them elsewhere. PENDING_CLOSEs may be a bit more tricky, but there must perhaps at least be a way to give up permanently on a movement plan, and letting things be for a while hoping for the RS to recover itself on its own (such that clients also have a chance of getting things to work in the meantime)? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11307) Deprecate SlabCache
[ https://issues.apache.org/jira/browse/HBASE-11307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11307: - Attachment: HBASE-11307.01-0.98.patch A patch for 0.98 as well. This one excludes the docbook updates. Deprecate SlabCache --- Key: HBASE-11307 URL: https://issues.apache.org/jira/browse/HBASE-11307 Project: HBase Issue Type: Sub-task Components: regionserver Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 0.99.0, 0.98.4 Attachments: HBASE-11307.00.patch, HBASE-11307.01-0.98.patch, HBASE-11307.01.patch BucketCache is more feature-full than SlabCache. My performance testing indicates that SlabCache doesn't perform well as memory footprint grows. I believe BucketCache is seeing more production usage than SlabCache at this point. I suggest we consolidate on BucketCache. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11342) The method isChildReadLock in class ZKInterProcessLockBase is wrong
[
https://issues.apache.org/jira/browse/HBASE-11342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-11342:
--
Status: Patch Available (was: Open)
The method isChildReadLock in class ZKInterProcessLockBase is wrong
---
Key: HBASE-11342
URL: https://issues.apache.org/jira/browse/HBASE-11342
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 0.98.3, 0.99.0
Reporter: Qianxi Zhang
Assignee: Qianxi Zhang
Priority: Minor
Attachments: HBASE_11342.patch
The method isChildReadLock in class ZKInterProcessLockBase may be wrong,
which determines whether the lock is readLock or not. So we should compare
the node name with READ_LOCK_CHILD_NODE_PREFIX rather than
WRITE_LOCK_CHILD_NODE_PREFIX. Since there is no other method to invoke the
method isChildReadLock now, we have not encountered an error.
{code}
protected static boolean isChildReadLock(String child) {
int idx = child.lastIndexOf(ZKUtil.ZNODE_PATH_SEPARATOR);
String suffix = child.substring(idx + 1);
return suffix.startsWith(WRITE_LOCK_CHILD_NODE_PREFIX);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11342) The method isChildReadLock in class ZKInterProcessLockBase is wrong
[
https://issues.apache.org/jira/browse/HBASE-11342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030813#comment-14030813
]
stack commented on HBASE-11342:
---
Looks like a pretty basic fix to me. +1
The method isChildReadLock in class ZKInterProcessLockBase is wrong
---
Key: HBASE-11342
URL: https://issues.apache.org/jira/browse/HBASE-11342
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 0.99.0, 0.98.3
Reporter: Qianxi Zhang
Assignee: Qianxi Zhang
Priority: Minor
Attachments: HBASE_11342.patch
The method isChildReadLock in class ZKInterProcessLockBase may be wrong,
which determines whether the lock is readLock or not. So we should compare
the node name with READ_LOCK_CHILD_NODE_PREFIX rather than
WRITE_LOCK_CHILD_NODE_PREFIX. Since there is no other method to invoke the
method isChildReadLock now, we have not encountered an error.
{code}
protected static boolean isChildReadLock(String child) {
int idx = child.lastIndexOf(ZKUtil.ZNODE_PATH_SEPARATOR);
String suffix = child.substring(idx + 1);
return suffix.startsWith(WRITE_LOCK_CHILD_NODE_PREFIX);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11307) Deprecate SlabCache
[ https://issues.apache.org/jira/browse/HBASE-11307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030821#comment-14030821 ] stack commented on HBASE-11307: --- Go for it. Deprecate SlabCache --- Key: HBASE-11307 URL: https://issues.apache.org/jira/browse/HBASE-11307 Project: HBase Issue Type: Sub-task Components: regionserver Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 0.99.0, 0.98.4 Attachments: HBASE-11307.00.patch, HBASE-11307.01-0.98.patch, HBASE-11307.01.patch BucketCache is more feature-full than SlabCache. My performance testing indicates that SlabCache doesn't perform well as memory footprint grows. I believe BucketCache is seeing more production usage than SlabCache at this point. I suggest we consolidate on BucketCache. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11307) Deprecate SlabCache
[
https://issues.apache.org/jira/browse/HBASE-11307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030824#comment-14030824
]
Hadoop QA commented on HBASE-11307:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12650331/HBASE-11307.01-0.98.patch
against trunk revision .
ATTACHMENT ID: 12650331
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9769//console
This message is automatically generated.
Deprecate SlabCache
---
Key: HBASE-11307
URL: https://issues.apache.org/jira/browse/HBASE-11307
Project: HBase
Issue Type: Sub-task
Components: regionserver
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Fix For: 0.99.0, 0.98.4
Attachments: HBASE-11307.00.patch, HBASE-11307.01-0.98.patch,
HBASE-11307.01.patch
BucketCache is more feature-full than SlabCache. My performance testing
indicates that SlabCache doesn't perform well as memory footprint grows. I
believe BucketCache is seeing more production usage than SlabCache at this
point. I suggest we consolidate on BucketCache.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11340) Remove references to xcievers in documentation
[
https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030823#comment-14030823
]
stack commented on HBASE-11340:
---
Looks good. Suggest you keep more of the 'formerly known as xceivers' and
mention old config. When I apply the patch I get this:
{code}
kalashnikov:hbase.git.commit stack$ xmllint src/main/docbkx/configuration.xml
src/main/docbkx/configuration.xml:531: parser error : Opening and ending tag
mismatch: para line 521 and section
/section
^
src/main/docbkx/configuration.xml:1556: parser error : Opening and ending tag
mismatch: section line 54 and chapter
/chapter
^
src/main/docbkx/configuration.xml:1557: parser error : Premature end of data in
tag chapter line 2
^
{code}
Seems like a missing closer or something? If you need help figuring it, just
holler.
Remove references to xcievers in documentation
--
Key: HBASE-11340
URL: https://issues.apache.org/jira/browse/HBASE-11340
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Dima Spivak
Assignee: Dima Spivak
Priority: Trivial
Attachments: HBASE-11340.patch
While most references to xcievers were removed in HBase as part of
[HBASE-5697], the ref guide still contains a few stray references.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11307) Deprecate SlabCache
[ https://issues.apache.org/jira/browse/HBASE-11307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11307: - Resolution: Fixed Release Note: SlabCache is deprecated and will be dropped for 1.0! Start investigating BucketCache for your off-heap caching needs. Status: Resolved (was: Patch Available) Committed to master and 0.98. Thanks for the reviews. Deprecate SlabCache --- Key: HBASE-11307 URL: https://issues.apache.org/jira/browse/HBASE-11307 Project: HBase Issue Type: Sub-task Components: regionserver Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 0.99.0, 0.98.4 Attachments: HBASE-11307.00.patch, HBASE-11307.01-0.98.patch, HBASE-11307.01.patch BucketCache is more feature-full than SlabCache. My performance testing indicates that SlabCache doesn't perform well as memory footprint grows. I believe BucketCache is seeing more production usage than SlabCache at this point. I suggest we consolidate on BucketCache. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11340) Remove references to xcievers in documentation
[ https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030829#comment-14030829 ] Dima Spivak commented on HBASE-11340: - Oops, sorry about that. I'll fix the formatting issue, throw in a mention of the former name, and upload a new revision shortly. Thanks for taking a look! -Dima Remove references to xcievers in documentation -- Key: HBASE-11340 URL: https://issues.apache.org/jira/browse/HBASE-11340 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Assignee: Dima Spivak Priority: Trivial Attachments: HBASE-11340.patch While most references to xcievers were removed in HBase as part of [HBASE-5697], the ref guide still contains a few stray references. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11344) Hide row keys and such from the web UIs
Devaraj Das created HBASE-11344: --- Summary: Hide row keys and such from the web UIs Key: HBASE-11344 URL: https://issues.apache.org/jira/browse/HBASE-11344 Project: HBase Issue Type: Improvement Reporter: Devaraj Das Fix For: 0.99.0 The table details on the master UI lists the start row keys of the regions. The row keys might have sensitive data. We should hide them based on whether or not the user accessing has the required authorization to view the table.. To start with, we could make the display of row keys and such based on a configuration being true or false. If it is false, such potentially sensitive data is never displayed on the web UI. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11340) Remove references to xcievers in documentation
[ https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11340: Attachment: HBASE-11340-v2.patch Fixed the missing closing para tag and added a mention of the formerly known as name in the configuration section (along with a link to the blog post discussing it). Remove references to xcievers in documentation -- Key: HBASE-11340 URL: https://issues.apache.org/jira/browse/HBASE-11340 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Assignee: Dima Spivak Priority: Trivial Attachments: HBASE-11340-v2.patch, HBASE-11340.patch While most references to xcievers were removed in HBase as part of [HBASE-5697], the ref guide still contains a few stray references. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10871) Indefinite OPEN/CLOSE wait on busy RegionServers
[ https://issues.apache.org/jira/browse/HBASE-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030849#comment-14030849 ] Hudson commented on HBASE-10871: SUCCESS: Integrated in HBase-0.94-security #491 (See [https://builds.apache.org/job/HBase-0.94-security/491/]) HBASE-10871 Indefinite OPEN/CLOSE wait on busy RegionServers (Esteban) (jxiang: rev a228e925e7a80ae479700aa40b43ab2e002d4e7c) * src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java Indefinite OPEN/CLOSE wait on busy RegionServers Key: HBASE-10871 URL: https://issues.apache.org/jira/browse/HBASE-10871 Project: HBase Issue Type: Improvement Components: Balancer, master, Region Assignment Affects Versions: 0.94.6 Reporter: Harsh J Assignee: Esteban Gutierrez Fix For: 0.99.0, 0.94.21, 0.98.4 Attachments: HBASE-10871-0.94.v1.patch, HBASE-10871.v0.patch, HBASE-10871.v1.patch We observed a case where, when a specific RS got bombarded by a large amount of regular requests, spiking and filling up its RPC queue, the balancer's invoked unassigns and assigns for regions that dealt with this server entered into an indefinite retry loop. The regions specifically began waiting in PENDING_CLOSE/PENDING_OPEN states indefinitely cause of the HBase Client RPC from the ServerManager at the master was running into SocketTimeouts. This caused a region unavailability in the server for the affected regions. The timeout monitor retry default of 30m in 0.94's AM compounded the waiting gap further a bit more (this is now 10m in 0.95+'s new AM, and has further retries before we get there, which is good). Wonder if there's a way to improve this situation generally. PENDING_OPENs may be easy to handle - we can switch them out and move them elsewhere. PENDING_CLOSEs may be a bit more tricky, but there must perhaps at least be a way to give up permanently on a movement plan, and letting things be for a while hoping for the RS to recover itself on its own (such that clients also have a chance of getting things to work in the meantime)? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11094) Distributed log replay is incompatible for rolling restarts
[ https://issues.apache.org/jira/browse/HBASE-11094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030851#comment-14030851 ] stack commented on HBASE-11094: --- bq. Once 1.0 branch is cut, I'll create a JIRA to turn distributedLogReplay off by default depends on the rolling upgrade story for 1.0. What does the above mean [~jeffreyz]? Can you add a release note saying what has been implemented? I know you say '(basically no release note for the JIRA)' above but would be good for us devs trying to follow along (or who may have to debug it in future). Please describe what was implemented To be clear, mode is LOG_REPLAY currently in master? Reading over the patch, it looks good. I'm not sure I am clear on all that is going on but I like the sound of it (smile). Distributed log replay is incompatible for rolling restarts --- Key: HBASE-11094 URL: https://issues.apache.org/jira/browse/HBASE-11094 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Jeffrey Zhong Priority: Blocker Fix For: 0.99.0 Attachments: hbase-11094-v2.patch, hbase-11094-v3.patch, hbase-11094-v4.patch, hbase-11094-v5.1.patch, hbase-11094-v5.patch, hbase-11094.patch 0.99.0 comes with dist log replay by default (HBASE-10888). However, reading the code and discussing this with Jeffrey, we realized that the dist log replay code is not compatible with rolling upgrades from 0.98.0 and 1.0.0. The issue is that, the region server looks at it own configuration to decide whether the region should be opened in replay mode or not. The open region RPC does not contain that info. So if dist log replay is enabled on master, the master will assign the region and schedule replay tasks. If the region is opened in a RS that does not have this conf enabled, then it will happily open the region in normal mode (not replay mode) causing possible (transient) data loss. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11340) Remove references to xcievers in documentation
[ https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11340: -- Resolution: Fixed Fix Version/s: 0.99.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks for the patch [~dimaspivak] Remove references to xcievers in documentation -- Key: HBASE-11340 URL: https://issues.apache.org/jira/browse/HBASE-11340 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Assignee: Dima Spivak Priority: Trivial Fix For: 0.99.0 Attachments: HBASE-11340-v2.patch, HBASE-11340.patch While most references to xcievers were removed in HBase as part of [HBASE-5697], the ref guide still contains a few stray references. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11345) Add an option not to restore cluster after an IT test
Jimmy Xiang created HBASE-11345: --- Summary: Add an option not to restore cluster after an IT test Key: HBASE-11345 URL: https://issues.apache.org/jira/browse/HBASE-11345 Project: HBase Issue Type: Test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor After some IT test failure, we have to dig from logs to see what's wrong if we restore the cluster. It will be good if we can keep the current state of the cluster for investigation. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10923) Control where to put meta region
[ https://issues.apache.org/jira/browse/HBASE-10923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030865#comment-14030865 ] stack commented on HBASE-10923: --- [~jxiang] Do you want to release note hbase.balancer.tablesOnMaster? It is off by default currently? Control where to put meta region Key: HBASE-10923 URL: https://issues.apache.org/jira/browse/HBASE-10923 Project: HBase Issue Type: Improvement Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.99.0 Attachments: hbase-10923.patch There is a concern on placing meta regions on the master, as in the comments of HBASE-10569. I was thinking we should have a configuration for a load balancer to decide where to put it. Adjusting this configuration we can control whether to put the meta on master, or other region server. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11337) Document how to create, modify, delete a table using Java
[
https://issues.apache.org/jira/browse/HBASE-11337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-11337:
--
Resolution: Fixed
Fix Version/s: 0.99.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed for now. Thanks Misty.
Example code, especially java code like this, should be part of the build
process so it breaks if the APIs change. Without that, code like this that
just lives in the doc will only rot.
Document how to create, modify, delete a table using Java
-
Key: HBASE-11337
URL: https://issues.apache.org/jira/browse/HBASE-11337
Project: HBase
Issue Type: Bug
Components: Admin, documentation
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
Fix For: 0.99.0
Attachments: HBASE-11337.patch
Example code from [~jmspaggi]
{code}
package com.example.hbase.admin;
import java.io.IOException;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.io.compress.Compression.Algorithm;
import org.apache.hadoop.conf.Configuration;
import static com.example.hbase.Constants.*;
public class CreateSchema {
public static void createIfNotExist(HBaseAdmin admin, HTableDescriptor
table) throws IOException {
if (admin.tableExists(table.getName())) {
admin.disableTable(table.getName());
admin.deleteTable(table.getName());
}
admin.createTable(table);
}
public static void main(String[] args) {
/**/
/* Create application schema. */
/**/
Configuration config = HBaseConfiguration.create();
config.set(hbase.zookeeper.quorum, 192.168.56.102); // Here we are
running zookeeper locally
try {
final HBaseAdmin admin = new HBaseAdmin(config);
HTableDescriptor table_assetmeta = new
HTableDescriptor(TableName.valueOf(TABLE_ASSETMETA));
table_assetmeta.addFamily(new
HColumnDescriptor(CF_DEFAULT).setCompressionType(Algorithm.GZ));
System.out.print(Creating table_assetmeta. );
admin.createTable(table_assetmeta);
System.out.println( Done.);
admin.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11346) Enable cacheonwrite for index and bloom blocks by default
Nick Dimiduk created HBASE-11346: Summary: Enable cacheonwrite for index and bloom blocks by default Key: HBASE-11346 URL: https://issues.apache.org/jira/browse/HBASE-11346 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.99.0, 0.98.4 Reporter: Nick Dimiduk Theory is, this will reduce the cost of compactions on a mixed workload cluster, reducing the number of seeks for the first blocks from the new HFile from ~4 to ~1. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11347) For some errors, the client can retry infinitely
Nicolas Liochon created HBASE-11347: --- Summary: For some errors, the client can retry infinitely Key: HBASE-11347 URL: https://issues.apache.org/jira/browse/HBASE-11347 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.3, 0.99.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Critical Fix For: 0.99.0, 0.98.4 As the title says... -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11347) For some errors, the client can retry infinitely
[ https://issues.apache.org/jira/browse/HBASE-11347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-11347: Attachment: 11347.v1.patch For some errors, the client can retry infinitely Key: HBASE-11347 URL: https://issues.apache.org/jira/browse/HBASE-11347 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.99.0, 0.98.3 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Critical Fix For: 0.99.0, 0.98.4 Attachments: 11347.v1.patch As the title says... -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11347) For some errors, the client can retry infinitely
[ https://issues.apache.org/jira/browse/HBASE-11347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-11347: Status: Patch Available (was: Open) For some errors, the client can retry infinitely Key: HBASE-11347 URL: https://issues.apache.org/jira/browse/HBASE-11347 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.3, 0.99.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Critical Fix For: 0.99.0, 0.98.4 Attachments: 11347.v1.patch As the title says... -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11347) For some errors, the client can retry infinitely
[ https://issues.apache.org/jira/browse/HBASE-11347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030909#comment-14030909 ] Nicolas Liochon commented on HBASE-11347: - The hard part was to write to the test correctly... I will do the 98 patch once this fix is reviewed. For some errors, the client can retry infinitely Key: HBASE-11347 URL: https://issues.apache.org/jira/browse/HBASE-11347 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.99.0, 0.98.3 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Critical Fix For: 0.99.0, 0.98.4 Attachments: 11347.v1.patch As the title says... -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-8315) Documentation should have more information of LRU Stats
[
https://issues.apache.org/jira/browse/HBASE-8315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-8315:
-
Attachment: 8351.txt
I just put [~daisuke.kobayashi]'s suggested text into the UI.
Documentation should have more information of LRU Stats
---
Key: HBASE-8315
URL: https://issues.apache.org/jira/browse/HBASE-8315
Project: HBase
Issue Type: Improvement
Components: documentation
Affects Versions: 0.95.0
Reporter: Daisuke Kobayashi
Assignee: Misty Stanley-Jones
Labels: documentation
Attachments: 8351.txt
Unfortunately, there's no documentation to explain the meaning of each LRU
Stats in the regionserver logs. So this is for creating a new paragraph
regarding this. My current idea is below, but it's a little bit difficult to
explain the difference between 'cachingAccesses' and 'accesses' from an
administrator or a user views.
Could you guys help to improve the content?
{noformat}
total: The current memory size of the cache in use.
free: The total free memory currently available to store more cache entries.
max: Maximum allowed memory size of the cache.
blocks: Caches store blocks of data; this number is the current # of blocks
stored, which use up the total memory space.
accesses: The total number of times the cache was accessed, regardless of
result.
hits: The total number of times the cache was accessed and the result was a
successful hit (presence of looked up element in cache is a hit).
hitRatio: The current percentage for hits / accesses.
Unclear:
cachingAccesses: cachingHits + The number of getBlock requests that were
cache misses, but only from requests that were set to use the block cache.
cachingHits: The number of getBlock requests that were cache hits, but only
from requests that were set to use the block cache. This is because all reads
=
cachingHitsRatio: The current percentage for cachintHits / cachingAccesses
evictions: The total number of times an eviction has occurred (based on the
use of the LRU algorithm)
evicted: The total number of blocks that have been evicted (based on the use
of the LRU algorithm)
evictedPerRun: The total number of blocks that have been evicted overall /
The number of times an eviction has occurred overall
{noformat}
And also, where should we add this paragraph in the documentation?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-8315) Documentation should have more information of LRU Stats
[
https://issues.apache.org/jira/browse/HBASE-8315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-8315:
-
Attachment: Screen Shot 2014-06-13 at 10.59.05 AM.png
Screen shot of what it looks like in the UI
Documentation should have more information of LRU Stats
---
Key: HBASE-8315
URL: https://issues.apache.org/jira/browse/HBASE-8315
Project: HBase
Issue Type: Improvement
Components: documentation
Affects Versions: 0.95.0
Reporter: Daisuke Kobayashi
Assignee: Misty Stanley-Jones
Labels: documentation
Attachments: 8351.txt, Screen Shot 2014-06-13 at 10.59.05 AM.png
Unfortunately, there's no documentation to explain the meaning of each LRU
Stats in the regionserver logs. So this is for creating a new paragraph
regarding this. My current idea is below, but it's a little bit difficult to
explain the difference between 'cachingAccesses' and 'accesses' from an
administrator or a user views.
Could you guys help to improve the content?
{noformat}
total: The current memory size of the cache in use.
free: The total free memory currently available to store more cache entries.
max: Maximum allowed memory size of the cache.
blocks: Caches store blocks of data; this number is the current # of blocks
stored, which use up the total memory space.
accesses: The total number of times the cache was accessed, regardless of
result.
hits: The total number of times the cache was accessed and the result was a
successful hit (presence of looked up element in cache is a hit).
hitRatio: The current percentage for hits / accesses.
Unclear:
cachingAccesses: cachingHits + The number of getBlock requests that were
cache misses, but only from requests that were set to use the block cache.
cachingHits: The number of getBlock requests that were cache hits, but only
from requests that were set to use the block cache. This is because all reads
=
cachingHitsRatio: The current percentage for cachintHits / cachingAccesses
evictions: The total number of times an eviction has occurred (based on the
use of the LRU algorithm)
evicted: The total number of blocks that have been evicted (based on the use
of the LRU algorithm)
evictedPerRun: The total number of blocks that have been evicted overall /
The number of times an eviction has occurred overall
{noformat}
And also, where should we add this paragraph in the documentation?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HBASE-8315) Documentation should have more information of LRU Stats
[
https://issues.apache.org/jira/browse/HBASE-8315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-8315.
--
Resolution: Fixed
Fix Version/s: 0.99.0
Committed small text change in UI and committed to master.
Documentation should have more information of LRU Stats
---
Key: HBASE-8315
URL: https://issues.apache.org/jira/browse/HBASE-8315
Project: HBase
Issue Type: Improvement
Components: documentation
Affects Versions: 0.95.0
Reporter: Daisuke Kobayashi
Assignee: Misty Stanley-Jones
Labels: documentation
Fix For: 0.99.0
Attachments: 8351.txt, Screen Shot 2014-06-13 at 10.59.05 AM.png
Unfortunately, there's no documentation to explain the meaning of each LRU
Stats in the regionserver logs. So this is for creating a new paragraph
regarding this. My current idea is below, but it's a little bit difficult to
explain the difference between 'cachingAccesses' and 'accesses' from an
administrator or a user views.
Could you guys help to improve the content?
{noformat}
total: The current memory size of the cache in use.
free: The total free memory currently available to store more cache entries.
max: Maximum allowed memory size of the cache.
blocks: Caches store blocks of data; this number is the current # of blocks
stored, which use up the total memory space.
accesses: The total number of times the cache was accessed, regardless of
result.
hits: The total number of times the cache was accessed and the result was a
successful hit (presence of looked up element in cache is a hit).
hitRatio: The current percentage for hits / accesses.
Unclear:
cachingAccesses: cachingHits + The number of getBlock requests that were
cache misses, but only from requests that were set to use the block cache.
cachingHits: The number of getBlock requests that were cache hits, but only
from requests that were set to use the block cache. This is because all reads
=
cachingHitsRatio: The current percentage for cachintHits / cachingAccesses
evictions: The total number of times an eviction has occurred (based on the
use of the LRU algorithm)
evicted: The total number of blocks that have been evicted (based on the use
of the LRU algorithm)
evictedPerRun: The total number of blocks that have been evicted overall /
The number of times an eviction has occurred overall
{noformat}
And also, where should we add this paragraph in the documentation?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4089) blockCache contents report
[ https://issues.apache.org/jira/browse/HBASE-4089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030923#comment-14030923 ] stack commented on HBASE-4089: -- bq. Does this JIRA address or relate to https://issues.apache.org/jira/browse/HBASE-8315? [~misty] It does with some minor mods commtted against HBASE-8315 blockCache contents report -- Key: HBASE-4089 URL: https://issues.apache.org/jira/browse/HBASE-4089 Project: HBase Issue Type: New Feature Reporter: Doug Meil Assignee: stack Fix For: 0.99.0 Attachments: 4089.wip.txt, 4089.wip.v2.txt, 4089v3.txt, 4089v4.txt, 4089v6.txt, 4089v7.txt, 4089v7.txt, 4089v8.txt, bc_basic.png, bc_bucketcache.png, bc_buckets.png, bc_config.png, bc_l1.png, bc_l2_buckets.png, bc_showing_buckets_start.png, bc_slabs.png, bc_slabs.png, bc_stats.png, hbase_4089_blockcachereport.pdf, java_blockcache_checkpoint_2011_08_11.patch Summarized block-cache report for a RegionServer would be helpful. For example ... table1 cf1 100 blocks, totalBytes=y, averageTimeInCache= hours cf2 200 blocks, totalBytes=z, averageTimeInCache= hours table2 cf1 75 blocks, totalBytes=y, averageTimeInCache= hours cf2 150 blocks, totalBytes=z, averageTimeInCache= hours ... Etc. The current metrics list blockCacheSize and blockCacheFree, but there is no way to know what's in there. Any single block isn't really important, but the patterns of what CF/Table they came from, how big are they, and how long (on average) they've been in the cache, are important. No such interface exists in HRegionInterface. But I think it would be helpful from an operational perspective. Updated (7-29): Removing suggestion for UI. I would be happy just to get this report on a configured interval dumped to a log file. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10871) Indefinite OPEN/CLOSE wait on busy RegionServers
[ https://issues.apache.org/jira/browse/HBASE-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030929#comment-14030929 ] Hudson commented on HBASE-10871: SUCCESS: Integrated in HBase-0.94-JDK7 #145 (See [https://builds.apache.org/job/HBase-0.94-JDK7/145/]) HBASE-10871 Indefinite OPEN/CLOSE wait on busy RegionServers (Esteban) (jxiang: rev a228e925e7a80ae479700aa40b43ab2e002d4e7c) * src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java Indefinite OPEN/CLOSE wait on busy RegionServers Key: HBASE-10871 URL: https://issues.apache.org/jira/browse/HBASE-10871 Project: HBase Issue Type: Improvement Components: Balancer, master, Region Assignment Affects Versions: 0.94.6 Reporter: Harsh J Assignee: Esteban Gutierrez Fix For: 0.99.0, 0.94.21, 0.98.4 Attachments: HBASE-10871-0.94.v1.patch, HBASE-10871.v0.patch, HBASE-10871.v1.patch We observed a case where, when a specific RS got bombarded by a large amount of regular requests, spiking and filling up its RPC queue, the balancer's invoked unassigns and assigns for regions that dealt with this server entered into an indefinite retry loop. The regions specifically began waiting in PENDING_CLOSE/PENDING_OPEN states indefinitely cause of the HBase Client RPC from the ServerManager at the master was running into SocketTimeouts. This caused a region unavailability in the server for the affected regions. The timeout monitor retry default of 30m in 0.94's AM compounded the waiting gap further a bit more (this is now 10m in 0.95+'s new AM, and has further retries before we get there, which is good). Wonder if there's a way to improve this situation generally. PENDING_OPENs may be easy to handle - we can switch them out and move them elsewhere. PENDING_CLOSEs may be a bit more tricky, but there must perhaps at least be a way to give up permanently on a movement plan, and letting things be for a while hoping for the RS to recover itself on its own (such that clients also have a chance of getting things to work in the meantime)? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-6192) Document ACL matrix in the book
[ https://issues.apache.org/jira/browse/HBASE-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030926#comment-14030926 ] Matteo Bertozzi commented on HBASE-6192: I don't think the pdf reflects the current status, I took a quick look and stuff like bulk load, flush, compact (which were recently changed) are marked as ADMIN instead of CREATE/ADMIN Document ACL matrix in the book --- Key: HBASE-6192 URL: https://issues.apache.org/jira/browse/HBASE-6192 Project: HBase Issue Type: Task Components: documentation, security Affects Versions: 0.94.1, 0.95.2 Reporter: Enis Soztutar Assignee: Misty Stanley-Jones Labels: documentaion, security Fix For: 0.99.0 Attachments: HBASE-6192-rebased.patch, HBASE-6192.patch, HBase Security-ACL Matrix.pdf, HBase Security-ACL Matrix.pdf, HBase Security-ACL Matrix.pdf, HBase Security-ACL Matrix.xls, HBase Security-ACL Matrix.xls, HBase Security-ACL Matrix.xls We have an excellent matrix at https://issues.apache.org/jira/secure/attachment/12531252/Security-ACL%20Matrix.pdf for ACL. Once the changes are done, we can adapt that and put it in the book, also add some more documentation about the new authorization features. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10871) Indefinite OPEN/CLOSE wait on busy RegionServers
[ https://issues.apache.org/jira/browse/HBASE-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030935#comment-14030935 ] Hudson commented on HBASE-10871: SUCCESS: Integrated in HBase-0.98 #334 (See [https://builds.apache.org/job/HBase-0.98/334/]) HBASE-10871 Indefinite OPEN/CLOSE wait on busy RegionServers (Esteban) (jxiang: rev 7ffc454ccc64f095d8992f03edeb3aacd83de92e) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java Indefinite OPEN/CLOSE wait on busy RegionServers Key: HBASE-10871 URL: https://issues.apache.org/jira/browse/HBASE-10871 Project: HBase Issue Type: Improvement Components: Balancer, master, Region Assignment Affects Versions: 0.94.6 Reporter: Harsh J Assignee: Esteban Gutierrez Fix For: 0.99.0, 0.94.21, 0.98.4 Attachments: HBASE-10871-0.94.v1.patch, HBASE-10871.v0.patch, HBASE-10871.v1.patch We observed a case where, when a specific RS got bombarded by a large amount of regular requests, spiking and filling up its RPC queue, the balancer's invoked unassigns and assigns for regions that dealt with this server entered into an indefinite retry loop. The regions specifically began waiting in PENDING_CLOSE/PENDING_OPEN states indefinitely cause of the HBase Client RPC from the ServerManager at the master was running into SocketTimeouts. This caused a region unavailability in the server for the affected regions. The timeout monitor retry default of 30m in 0.94's AM compounded the waiting gap further a bit more (this is now 10m in 0.95+'s new AM, and has further retries before we get there, which is good). Wonder if there's a way to improve this situation generally. PENDING_OPENs may be easy to handle - we can switch them out and move them elsewhere. PENDING_CLOSEs may be a bit more tricky, but there must perhaps at least be a way to give up permanently on a movement plan, and letting things be for a while hoping for the RS to recover itself on its own (such that clients also have a chance of getting things to work in the meantime)? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11069) Decouple region merging from ZooKeeper
[
https://issues.apache.org/jira/browse/HBASE-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030937#comment-14030937
]
stack commented on HBASE-11069:
---
[~sergey.soldatov] nit, you should look at the below in generated javadoc...
+ * Coordination operations for region merge transaction. The operation should
be coordinated at the
+ * following stages:
+ * 1. startRegionMergeTransaction - all preparation/initialization for merge
region transaction
+ * 2. waitForRegionMergeTransaction - wait until coordination complete all
works related to merge
+ * 3. confirmRegionMergeTransaction - confirm that the merge could be
completed and none of
+ *merging regions moved somehow
+ * 4. completeRegionMergeTransaction - all steps that are required to complete
the transaction.
+ *Called after PONR (point of no return)
... to see how it displays (mangled).
That said, nice doc. Pity it doesn't diplay properly.
This works in java6?
+ /**
+ * Dummy interface for region merge transaction details.
+ */
+ public static interface RegionMergeDetails {
+ }
Otherwise patch looks good.
[~mantonov] You reviewed this?
Decouple region merging from ZooKeeper
--
Key: HBASE-11069
URL: https://issues.apache.org/jira/browse/HBASE-11069
Project: HBase
Issue Type: Sub-task
Components: Consensus, Zookeeper
Reporter: Sergey Soldatov
Assignee: Sergey Soldatov
Attachments: HBASE-11069.patch, HBASE_11069-v2.patch,
HBASE_11069-v3.patch, HBASE_11069-v4.patch
Region Merge should be decoupled from ZK.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11331) [blockcache] lazy block decompression
[
https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030948#comment-14030948
]
Vladimir Rodionov commented on HBASE-11331:
---
These two statements contradict each other:
{quote}
Maintaining data in its compressed form in the block cache will greatly
increase our effective blockcache size and should show a meaning improvement in
cache hit rates in well designed applications.
{quote}
{quote}
The idea here is to lazily decompress/decrypt blocks when they're consumed,
rather than as soon as they're pulled off of disk.
{quote}
You either keep blocks compressed in a cache and decompress them on demand (1)
or you decompress them lazily and keep them decompressed after that (2). What
does lazy decompression means in this case? If you cache blocks on reads only
(most of the time and default behavior) - there is no much sense in a lazy
decompression, because your block will be accessed immediately after it got
into the cache. Lazy decompression makes sense only if you cache blocks on
writes, but in this case (2) contradicts (1) as I mentioned already.
[blockcache] lazy block decompression
-
Key: HBASE-11331
URL: https://issues.apache.org/jira/browse/HBASE-11331
Project: HBase
Issue Type: Improvement
Components: regionserver
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Attachments: HBASE-11331.00.patch
Maintaining data in its compressed form in the block cache will greatly
increase our effective blockcache size and should show a meaning improvement
in cache hit rates in well designed applications. The idea here is to lazily
decompress/decrypt blocks when they're consumed, rather than as soon as
they're pulled off of disk.
This is related to but less invasive than HBASE-8894.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11342) The method isChildReadLock in class ZKInterProcessLockBase is wrong
[
https://issues.apache.org/jira/browse/HBASE-11342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030951#comment-14030951
]
Hadoop QA commented on HBASE-11342:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12650281/HBASE_11342.patch
against trunk revision .
ATTACHMENT ID: 12650281
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9768//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9768//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9768//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9768//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9768//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9768//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9768//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9768//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9768//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9768//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9768//console
This message is automatically generated.
The method isChildReadLock in class ZKInterProcessLockBase is wrong
---
Key: HBASE-11342
URL: https://issues.apache.org/jira/browse/HBASE-11342
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 0.99.0, 0.98.3
Reporter: Qianxi Zhang
Assignee: Qianxi Zhang
Priority: Minor
Attachments: HBASE_11342.patch
The method isChildReadLock in class ZKInterProcessLockBase may be wrong,
which determines whether the lock is readLock or not. So we should compare
the node name with READ_LOCK_CHILD_NODE_PREFIX rather than
WRITE_LOCK_CHILD_NODE_PREFIX. Since there is no other method to invoke the
method isChildReadLock now, we have not encountered an error.
{code}
protected static boolean isChildReadLock(String child) {
int idx = child.lastIndexOf(ZKUtil.ZNODE_PATH_SEPARATOR);
String suffix = child.substring(idx + 1);
return suffix.startsWith(WRITE_LOCK_CHILD_NODE_PREFIX);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10923) Control where to put meta region
[ https://issues.apache.org/jira/browse/HBASE-10923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030956#comment-14030956 ] Jimmy Xiang commented on HBASE-10923: - Yes, I just added a release note. It is on by default currently. We can change the default for 1.0 branch if needed. Control where to put meta region Key: HBASE-10923 URL: https://issues.apache.org/jira/browse/HBASE-10923 Project: HBase Issue Type: Improvement Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.99.0 Attachments: hbase-10923.patch There is a concern on placing meta regions on the master, as in the comments of HBASE-10569. I was thinking we should have a configuration for a load balancer to decide where to put it. Adjusting this configuration we can control whether to put the meta on master, or other region server. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10923) Control where to put meta region
[ https://issues.apache.org/jira/browse/HBASE-10923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-10923: Release Note: This patch introduced a new configuration hbase.balancer.tablesOnMaster to control what tables' regions should be put on the master by a load balancer. By default, we will put regions of table acl, namespace, and meta on master, i.e. the default configuration is the same as hbase:acl,hbase:namespace,hbase:meta. To put no region on the master, you need to set hbase.balancer.tablesOnMaster to instead of an empty string(the default will be used if it is empty). Control where to put meta region Key: HBASE-10923 URL: https://issues.apache.org/jira/browse/HBASE-10923 Project: HBase Issue Type: Improvement Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.99.0 Attachments: hbase-10923.patch There is a concern on placing meta regions on the master, as in the comments of HBASE-10569. I was thinking we should have a configuration for a load balancer to decide where to put it. Adjusting this configuration we can control whether to put the meta on master, or other region server. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11331) [blockcache] lazy block decompression
[ https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030968#comment-14030968 ] Nick Dimiduk commented on HBASE-11331: -- In this implementation, the decompressed block does not replace the compressed block in the cache. Decompression cost is paid on block access, every time. I need to profile the scanner path to ensure a single request is not decompressing the same block multiple times. For hot blocks, I expect this to result in increased CPU load vs decompressing it only once. For a more evenly distributed access pattern, this should greatly reduce the amount of disk seeks because more data is cached. I believe the latter use-case is more common. [blockcache] lazy block decompression - Key: HBASE-11331 URL: https://issues.apache.org/jira/browse/HBASE-11331 Project: HBase Issue Type: Improvement Components: regionserver Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-11331.00.patch Maintaining data in its compressed form in the block cache will greatly increase our effective blockcache size and should show a meaning improvement in cache hit rates in well designed applications. The idea here is to lazily decompress/decrypt blocks when they're consumed, rather than as soon as they're pulled off of disk. This is related to but less invasive than HBASE-8894. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11345) Add an option not to restore cluster after an IT test
[ https://issues.apache.org/jira/browse/HBASE-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-11345: Attachment: hbase-11345.patch Add an option not to restore cluster after an IT test - Key: HBASE-11345 URL: https://issues.apache.org/jira/browse/HBASE-11345 Project: HBase Issue Type: Test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: hbase-11345.patch After some IT test failure, we have to dig from logs to see what's wrong if we restore the cluster. It will be good if we can keep the current state of the cluster for investigation. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11345) Add an option not to restore cluster after an IT test
[ https://issues.apache.org/jira/browse/HBASE-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-11345: Status: Patch Available (was: Open) Add an option not to restore cluster after an IT test - Key: HBASE-11345 URL: https://issues.apache.org/jira/browse/HBASE-11345 Project: HBase Issue Type: Test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: hbase-11345.patch After some IT test failure, we have to dig from logs to see what's wrong if we restore the cluster. It will be good if we can keep the current state of the cluster for investigation. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11118) non environment variable solution for IllegalAccessError: class com.google.protobuf.ZeroCopyLiteralByteString cannot access its superclass com.google.protobuf.Literal
[ https://issues.apache.org/jira/browse/HBASE-8?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030982#comment-14030982 ] Jonathan Hsieh commented on HBASE-8: Has using an alternative protobuf java lib like protostuff been considered? With that we could likely have more influence and get performance-oriented changes int and eventually percolate it to other ecosystem projects. non environment variable solution for IllegalAccessError: class com.google.protobuf.ZeroCopyLiteralByteString cannot access its superclass com.google.protobuf.LiteralByteString -- Key: HBASE-8 URL: https://issues.apache.org/jira/browse/HBASE-8 Project: HBase Issue Type: Bug Affects Versions: 0.98.2 Reporter: André Kelpe Assignee: Andrew Purtell Priority: Blocker Fix For: 0.99.0, 0.98.4 Attachments: HBASE-8-0.98.patch.gz, HBASE-8-trunk.patch.gz, shade_attempt.patch I am running into the problem described in https://issues.apache.org/jira/browse/HBASE-10304, while trying to use a newer version within cascading.hbase (https://github.com/cascading/cascading.hbase). One of the features of cascading.hbase is that you can use it from lingual (http://www.cascading.org/projects/lingual/), our SQL layer for hadoop. lingual has a notion of providers, which are fat jars that we pull down dynamically at runtime. Those jars give users the ability to talk to any system or format from SQL. They are added to the classpath programmatically before we submit jobs to a hadoop cluster. Since lingual does not know upfront , which providers are going to be used in a given run, the HADOOP_CLASSPATH trick proposed in the JIRA above is really clunky and breaks the ease of use we had before. No other provider requires this right now. It would be great to have a programmatical way to fix this, when using fat jars. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11340) Remove references to xcievers in documentation
[
https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030987#comment-14030987
]
Hadoop QA commented on HBASE-11340:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12650335/HBASE-11340-v2.patch
against trunk revision .
ATTACHMENT ID: 12650335
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9770//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9770//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9770//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9770//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9770//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9770//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9770//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9770//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9770//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9770//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9770//console
This message is automatically generated.
Remove references to xcievers in documentation
--
Key: HBASE-11340
URL: https://issues.apache.org/jira/browse/HBASE-11340
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Dima Spivak
Assignee: Dima Spivak
Priority: Trivial
Fix For: 0.99.0
Attachments: HBASE-11340-v2.patch, HBASE-11340.patch
While most references to xcievers were removed in HBase as part of
[HBASE-5697], the ref guide still contains a few stray references.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11331) [blockcache] lazy block decompression
[ https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030986#comment-14030986 ] Vladimir Rodionov commented on HBASE-11331: --- [~ndimiduk], where do you keep decompressed blocks? In a fast on heap cache? You do not have to decompress block every time you access it - only once and all subsequent scanner.next will read from decompressed block. Sorry, I am not following you here. [blockcache] lazy block decompression - Key: HBASE-11331 URL: https://issues.apache.org/jira/browse/HBASE-11331 Project: HBase Issue Type: Improvement Components: regionserver Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-11331.00.patch Maintaining data in its compressed form in the block cache will greatly increase our effective blockcache size and should show a meaning improvement in cache hit rates in well designed applications. The idea here is to lazily decompress/decrypt blocks when they're consumed, rather than as soon as they're pulled off of disk. This is related to but less invasive than HBASE-8894. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030997#comment-14030997
]
Mikhail Antonov commented on HBASE-4495:
@stack would you prefer this version of patch on RB?
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Attachments: HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11069) Decouple region merging from ZooKeeper
[ https://issues.apache.org/jira/browse/HBASE-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030998#comment-14030998 ] Mikhail Antonov commented on HBASE-11069: - [~stack] not the latest one yet, let me take a look. Decouple region merging from ZooKeeper -- Key: HBASE-11069 URL: https://issues.apache.org/jira/browse/HBASE-11069 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Sergey Soldatov Assignee: Sergey Soldatov Attachments: HBASE-11069.patch, HBASE_11069-v2.patch, HBASE_11069-v3.patch, HBASE_11069-v4.patch Region Merge should be decoupled from ZK. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11331) [blockcache] lazy block decompression
[ https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031002#comment-14031002 ] Vladimir Rodionov commented on HBASE-11331: --- In my own tests I have seen performance drop from ~ 100K to 75K ops with compression on. This is with a custom LZ4 codec and YMMV. I think 20-25% penalty is not that big to justify uncompressed mode of operation. In many cases, having more data in a cache is much more important than peak performance. [blockcache] lazy block decompression - Key: HBASE-11331 URL: https://issues.apache.org/jira/browse/HBASE-11331 Project: HBase Issue Type: Improvement Components: regionserver Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-11331.00.patch Maintaining data in its compressed form in the block cache will greatly increase our effective blockcache size and should show a meaning improvement in cache hit rates in well designed applications. The idea here is to lazily decompress/decrypt blocks when they're consumed, rather than as soon as they're pulled off of disk. This is related to but less invasive than HBASE-8894. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-8484) Adding diagnostic messages when the RegionServer hists an IOException in RegionServerReport
[ https://issues.apache.org/jira/browse/HBASE-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manukranth Kolloju resolved HBASE-8484. --- Resolution: Fixed Adding diagnostic messages when the RegionServer hists an IOException in RegionServerReport --- Key: HBASE-8484 URL: https://issues.apache.org/jira/browse/HBASE-8484 Project: HBase Issue Type: Task Components: regionserver Affects Versions: 0.89-fb Reporter: Manukranth Kolloju Priority: Trivial Fix For: 0.89-fb This diff adds instrumentation in the case when we catch IOException in regionserver report. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11331) [blockcache] lazy block decompression
[ https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031006#comment-14031006 ] Nick Dimiduk commented on HBASE-11331: -- Decompressed blocks aren't stored anywhere in the patch posted. This patch, just as current master code, decompressed an HFileBlock into an on-heap ByteBuffer. There's no additional cache layer for decompressed blocks as it stands currently; they're decompressed, consumed, and thrown away. HFileReaderV2$AbstractScannerV2 keeps a reference to the current block, so a single GET shouldn't pay the cost of decompression multiple times, but I need to confirm this is true. [blockcache] lazy block decompression - Key: HBASE-11331 URL: https://issues.apache.org/jira/browse/HBASE-11331 Project: HBase Issue Type: Improvement Components: regionserver Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-11331.00.patch Maintaining data in its compressed form in the block cache will greatly increase our effective blockcache size and should show a meaning improvement in cache hit rates in well designed applications. The idea here is to lazily decompress/decrypt blocks when they're consumed, rather than as soon as they're pulled off of disk. This is related to but less invasive than HBASE-8894. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10871) Indefinite OPEN/CLOSE wait on busy RegionServers
[ https://issues.apache.org/jira/browse/HBASE-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031015#comment-14031015 ] Hudson commented on HBASE-10871: SUCCESS: Integrated in HBase-0.94 #1377 (See [https://builds.apache.org/job/HBase-0.94/1377/]) HBASE-10871 Indefinite OPEN/CLOSE wait on busy RegionServers (Esteban) (jxiang: rev a228e925e7a80ae479700aa40b43ab2e002d4e7c) * src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java Indefinite OPEN/CLOSE wait on busy RegionServers Key: HBASE-10871 URL: https://issues.apache.org/jira/browse/HBASE-10871 Project: HBase Issue Type: Improvement Components: Balancer, master, Region Assignment Affects Versions: 0.94.6 Reporter: Harsh J Assignee: Esteban Gutierrez Fix For: 0.99.0, 0.94.21, 0.98.4 Attachments: HBASE-10871-0.94.v1.patch, HBASE-10871.v0.patch, HBASE-10871.v1.patch We observed a case where, when a specific RS got bombarded by a large amount of regular requests, spiking and filling up its RPC queue, the balancer's invoked unassigns and assigns for regions that dealt with this server entered into an indefinite retry loop. The regions specifically began waiting in PENDING_CLOSE/PENDING_OPEN states indefinitely cause of the HBase Client RPC from the ServerManager at the master was running into SocketTimeouts. This caused a region unavailability in the server for the affected regions. The timeout monitor retry default of 30m in 0.94's AM compounded the waiting gap further a bit more (this is now 10m in 0.95+'s new AM, and has further retries before we get there, which is good). Wonder if there's a way to improve this situation generally. PENDING_OPENs may be easy to handle - we can switch them out and move them elsewhere. PENDING_CLOSEs may be a bit more tricky, but there must perhaps at least be a way to give up permanently on a movement plan, and letting things be for a while hoping for the RS to recover itself on its own (such that clients also have a chance of getting things to work in the meantime)? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11331) [blockcache] lazy block decompression
[ https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031009#comment-14031009 ] Nick Dimiduk commented on HBASE-11331: -- bq. In many cases, having more data in a cache is much more important than peak performance. We are in complete agreement on this point. [blockcache] lazy block decompression - Key: HBASE-11331 URL: https://issues.apache.org/jira/browse/HBASE-11331 Project: HBase Issue Type: Improvement Components: regionserver Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-11331.00.patch Maintaining data in its compressed form in the block cache will greatly increase our effective blockcache size and should show a meaning improvement in cache hit rates in well designed applications. The idea here is to lazily decompress/decrypt blocks when they're consumed, rather than as soon as they're pulled off of disk. This is related to but less invasive than HBASE-8894. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-8844) Document the removal of replication state AKA start/stop_replication
[ https://issues.apache.org/jira/browse/HBASE-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031039#comment-14031039 ] Hudson commented on HBASE-8844: --- FAILURE: Integrated in HBase-TRUNK #5201 (See [https://builds.apache.org/job/HBase-TRUNK/5201/]) HBASE-8844 Document the removal of replication state AKA start/stop_replication (Misty Stanley-Jones) (jdcryans: rev 8e547f3ba73b698fc61d06d54b71fa4baabb0244) * hbase-server/src/main/javadoc/org/apache/hadoop/hbase/replication/package.html * src/main/site/xdoc/replication.xml Document the removal of replication state AKA start/stop_replication Key: HBASE-8844 URL: https://issues.apache.org/jira/browse/HBASE-8844 Project: HBase Issue Type: Task Components: documentation Reporter: Patrick Assignee: Misty Stanley-Jones Priority: Minor Fix For: 0.99.0 Attachments: HBASE-8844-v2.patch, HBASE-8844-v3-rebased.patch, HBASE-8844-v3.patch, HBASE-8844.patch The first two tutorials for enabling replication that google gives me [1], [2] take very different tones with regard to stop_replication. The HBase docs [1] make it sound fine to start and stop replication as desired. The Cloudera docs [2] say it may cause data loss. Which is true? If data loss is possible, are we talking about data loss in the primary cluster, or data loss in the standby cluster (presumably would require reinitializing the sync with a new CopyTable). [1] http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/replication/package-summary.html#requirements [2] http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/cdh4ig_topic_20_11.html -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10871) Indefinite OPEN/CLOSE wait on busy RegionServers
[ https://issues.apache.org/jira/browse/HBASE-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031038#comment-14031038 ] Hudson commented on HBASE-10871: FAILURE: Integrated in HBase-TRUNK #5201 (See [https://builds.apache.org/job/HBase-TRUNK/5201/]) HBASE-10871 Indefinite OPEN/CLOSE wait on busy RegionServers (Esteban) (jxiang: rev 92b494df459d0e3bbbda808c64889471cf915078) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java Indefinite OPEN/CLOSE wait on busy RegionServers Key: HBASE-10871 URL: https://issues.apache.org/jira/browse/HBASE-10871 Project: HBase Issue Type: Improvement Components: Balancer, master, Region Assignment Affects Versions: 0.94.6 Reporter: Harsh J Assignee: Esteban Gutierrez Fix For: 0.99.0, 0.94.21, 0.98.4 Attachments: HBASE-10871-0.94.v1.patch, HBASE-10871.v0.patch, HBASE-10871.v1.patch We observed a case where, when a specific RS got bombarded by a large amount of regular requests, spiking and filling up its RPC queue, the balancer's invoked unassigns and assigns for regions that dealt with this server entered into an indefinite retry loop. The regions specifically began waiting in PENDING_CLOSE/PENDING_OPEN states indefinitely cause of the HBase Client RPC from the ServerManager at the master was running into SocketTimeouts. This caused a region unavailability in the server for the affected regions. The timeout monitor retry default of 30m in 0.94's AM compounded the waiting gap further a bit more (this is now 10m in 0.95+'s new AM, and has further retries before we get there, which is good). Wonder if there's a way to improve this situation generally. PENDING_OPENs may be easy to handle - we can switch them out and move them elsewhere. PENDING_CLOSEs may be a bit more tricky, but there must perhaps at least be a way to give up permanently on a movement plan, and letting things be for a while hoping for the RS to recover itself on its own (such that clients also have a chance of getting things to work in the meantime)? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11118) non environment variable solution for IllegalAccessError: class com.google.protobuf.ZeroCopyLiteralByteString cannot access its superclass com.google.protobuf.Literal
[ https://issues.apache.org/jira/browse/HBASE-8?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031044#comment-14031044 ] Andrew Purtell commented on HBASE-8: No. Sounds interesting. Do we have a volunteer to replace our use of protobuf with protostuff (or whatever) and benchmark the performance delta under load on a cluster? Not sure I can find the time for that. Meanwhile some downstreamers like cascading are not able to use 0.98 or recent 0.96. non environment variable solution for IllegalAccessError: class com.google.protobuf.ZeroCopyLiteralByteString cannot access its superclass com.google.protobuf.LiteralByteString -- Key: HBASE-8 URL: https://issues.apache.org/jira/browse/HBASE-8 Project: HBase Issue Type: Bug Affects Versions: 0.98.2 Reporter: André Kelpe Assignee: Andrew Purtell Priority: Blocker Fix For: 0.99.0, 0.98.4 Attachments: HBASE-8-0.98.patch.gz, HBASE-8-trunk.patch.gz, shade_attempt.patch I am running into the problem described in https://issues.apache.org/jira/browse/HBASE-10304, while trying to use a newer version within cascading.hbase (https://github.com/cascading/cascading.hbase). One of the features of cascading.hbase is that you can use it from lingual (http://www.cascading.org/projects/lingual/), our SQL layer for hadoop. lingual has a notion of providers, which are fat jars that we pull down dynamically at runtime. Those jars give users the ability to talk to any system or format from SQL. They are added to the classpath programmatically before we submit jobs to a hadoop cluster. Since lingual does not know upfront , which providers are going to be used in a given run, the HADOOP_CLASSPATH trick proposed in the JIRA above is really clunky and breaks the ease of use we had before. No other provider requires this right now. It would be great to have a programmatical way to fix this, when using fat jars. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11118) non environment variable solution for IllegalAccessError: class com.google.protobuf.ZeroCopyLiteralByteString cannot access its superclass com.google.protobuf.Literal
[ https://issues.apache.org/jira/browse/HBASE-8?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031049#comment-14031049 ] Andrew Purtell commented on HBASE-8: How do we sync our use of google's protoc in the compile-protobuf build target with an alternative runtime? I'm not familiar with protostuff. Does it have it's own compiler ? non environment variable solution for IllegalAccessError: class com.google.protobuf.ZeroCopyLiteralByteString cannot access its superclass com.google.protobuf.LiteralByteString -- Key: HBASE-8 URL: https://issues.apache.org/jira/browse/HBASE-8 Project: HBase Issue Type: Bug Affects Versions: 0.98.2 Reporter: André Kelpe Assignee: Andrew Purtell Priority: Blocker Fix For: 0.99.0, 0.98.4 Attachments: HBASE-8-0.98.patch.gz, HBASE-8-trunk.patch.gz, shade_attempt.patch I am running into the problem described in https://issues.apache.org/jira/browse/HBASE-10304, while trying to use a newer version within cascading.hbase (https://github.com/cascading/cascading.hbase). One of the features of cascading.hbase is that you can use it from lingual (http://www.cascading.org/projects/lingual/), our SQL layer for hadoop. lingual has a notion of providers, which are fat jars that we pull down dynamically at runtime. Those jars give users the ability to talk to any system or format from SQL. They are added to the classpath programmatically before we submit jobs to a hadoop cluster. Since lingual does not know upfront , which providers are going to be used in a given run, the HADOOP_CLASSPATH trick proposed in the JIRA above is really clunky and breaks the ease of use we had before. No other provider requires this right now. It would be great to have a programmatical way to fix this, when using fat jars. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11118) non environment variable solution for IllegalAccessError: class com.google.protobuf.ZeroCopyLiteralByteString cannot access its superclass com.google.protobuf.Literal
[ https://issues.apache.org/jira/browse/HBASE-8?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031053#comment-14031053 ] stack commented on HBASE-8: --- [~tlipcon] bq. Have we already considered using our own classloader to isolate our dependency? There is a bit of messy research done at the head of HBASE-10304. IIRC, we don't get chance to do interception. RunJar is running the show. That is the case here too [~fs111]? Are you using hadoop runjar or a similar mechanism? bq. Instead, could we put some more pressure on the upstream protobuf maintainers to include the ZeroCopyLiteralByteString, or at least make its super-class non-final in order to support this? https://code.google.com/p/protobuf/issues/detail?id=374 asks and gets shutdown because it would break base precept that all is immutable in pb. bq. Alternatively, could we get shading to work by adding an extra indirection package that builds a jar-with-dependencies of both hbase-protocol and protobuf, and then shades that and re-publishes as an hbase-protocol-with-shaded-pb pom? Then the rest of HBase could depend on that? Let me try it Todd. non environment variable solution for IllegalAccessError: class com.google.protobuf.ZeroCopyLiteralByteString cannot access its superclass com.google.protobuf.LiteralByteString -- Key: HBASE-8 URL: https://issues.apache.org/jira/browse/HBASE-8 Project: HBase Issue Type: Bug Affects Versions: 0.98.2 Reporter: André Kelpe Assignee: Andrew Purtell Priority: Blocker Fix For: 0.99.0, 0.98.4 Attachments: HBASE-8-0.98.patch.gz, HBASE-8-trunk.patch.gz, shade_attempt.patch I am running into the problem described in https://issues.apache.org/jira/browse/HBASE-10304, while trying to use a newer version within cascading.hbase (https://github.com/cascading/cascading.hbase). One of the features of cascading.hbase is that you can use it from lingual (http://www.cascading.org/projects/lingual/), our SQL layer for hadoop. lingual has a notion of providers, which are fat jars that we pull down dynamically at runtime. Those jars give users the ability to talk to any system or format from SQL. They are added to the classpath programmatically before we submit jobs to a hadoop cluster. Since lingual does not know upfront , which providers are going to be used in a given run, the HADOOP_CLASSPATH trick proposed in the JIRA above is really clunky and breaks the ease of use we had before. No other provider requires this right now. It would be great to have a programmatical way to fix this, when using fat jars. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11342) The method isChildReadLock in class ZKInterProcessLockBase is wrong
[
https://issues.apache.org/jira/browse/HBASE-11342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-11342:
--
Resolution: Fixed
Fix Version/s: 0.98.4
0.99.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to master and to 0.98. Thank you for the patch [~qianxiZhang]
The method isChildReadLock in class ZKInterProcessLockBase is wrong
---
Key: HBASE-11342
URL: https://issues.apache.org/jira/browse/HBASE-11342
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 0.99.0, 0.98.3
Reporter: Qianxi Zhang
Assignee: Qianxi Zhang
Priority: Minor
Fix For: 0.99.0, 0.98.4
Attachments: HBASE_11342.patch
The method isChildReadLock in class ZKInterProcessLockBase may be wrong,
which determines whether the lock is readLock or not. So we should compare
the node name with READ_LOCK_CHILD_NODE_PREFIX rather than
WRITE_LOCK_CHILD_NODE_PREFIX. Since there is no other method to invoke the
method isChildReadLock now, we have not encountered an error.
{code}
protected static boolean isChildReadLock(String child) {
int idx = child.lastIndexOf(ZKUtil.ZNODE_PATH_SEPARATOR);
String suffix = child.substring(idx + 1);
return suffix.startsWith(WRITE_LOCK_CHILD_NODE_PREFIX);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11118) non environment variable solution for IllegalAccessError: class com.google.protobuf.ZeroCopyLiteralByteString cannot access its superclass com.google.protobuf.Literal
[ https://issues.apache.org/jira/browse/HBASE-8?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031064#comment-14031064 ] stack commented on HBASE-8: --- I was looking at protostuff last night. Seems dead but for a few recent commits and no mention of pb 2.5. [~ryanobjc] or [~posix4e]? Any input? non environment variable solution for IllegalAccessError: class com.google.protobuf.ZeroCopyLiteralByteString cannot access its superclass com.google.protobuf.LiteralByteString -- Key: HBASE-8 URL: https://issues.apache.org/jira/browse/HBASE-8 Project: HBase Issue Type: Bug Affects Versions: 0.98.2 Reporter: André Kelpe Assignee: Andrew Purtell Priority: Blocker Fix For: 0.99.0, 0.98.4 Attachments: HBASE-8-0.98.patch.gz, HBASE-8-trunk.patch.gz, shade_attempt.patch I am running into the problem described in https://issues.apache.org/jira/browse/HBASE-10304, while trying to use a newer version within cascading.hbase (https://github.com/cascading/cascading.hbase). One of the features of cascading.hbase is that you can use it from lingual (http://www.cascading.org/projects/lingual/), our SQL layer for hadoop. lingual has a notion of providers, which are fat jars that we pull down dynamically at runtime. Those jars give users the ability to talk to any system or format from SQL. They are added to the classpath programmatically before we submit jobs to a hadoop cluster. Since lingual does not know upfront , which providers are going to be used in a given run, the HADOOP_CLASSPATH trick proposed in the JIRA above is really clunky and breaks the ease of use we had before. No other provider requires this right now. It would be great to have a programmatical way to fix this, when using fat jars. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11345) Add an option not to restore cluster after an IT test
[ https://issues.apache.org/jira/browse/HBASE-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031076#comment-14031076 ] stack commented on HBASE-11345: --- lgtm Add an option not to restore cluster after an IT test - Key: HBASE-11345 URL: https://issues.apache.org/jira/browse/HBASE-11345 Project: HBase Issue Type: Test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: hbase-11345.patch After some IT test failure, we have to dig from logs to see what's wrong if we restore the cluster. It will be good if we can keep the current state of the cluster for investigation. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11069) Decouple region merging from ZooKeeper
[ https://issues.apache.org/jira/browse/HBASE-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031074#comment-14031074 ] Mikhail Antonov commented on HBASE-11069: - Looks good to me. I would be +1 - just a few tiny nits: - ZkRegionMergeCoordination and AssignementManger have unnecessary import, TestMasterFailover has unnecessary typecast - in the javadoc of RegionMergeCoordination may worth to add notes pointing where (Rs or master) each of key methods execute within. Decouple region merging from ZooKeeper -- Key: HBASE-11069 URL: https://issues.apache.org/jira/browse/HBASE-11069 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Sergey Soldatov Assignee: Sergey Soldatov Attachments: HBASE-11069.patch, HBASE_11069-v2.patch, HBASE_11069-v3.patch, HBASE_11069-v4.patch Region Merge should be decoupled from ZK. -- This message was sent by Atlassian JIRA (v6.2#6252)
