[jira] [Commented] (HBASE-15477) Do not save 'next block header' when we cache hfileblocks

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205844#comment-15205844
 ] 

stack commented on HBASE-15477:
---

Any +1s hereabouts (smile). I have questions on MBB but not really related to 
the patch. Thanks lads.

> Do not save 'next block header' when we cache hfileblocks
> -
>
> Key: HBASE-15477
> URL: https://issues.apache.org/jira/browse/HBASE-15477
> Project: HBase
>  Issue Type: Sub-task
>  Components: BlockCache, Performance
>Reporter: stack
>Assignee: stack
> Attachments: 15366v4.patch, 15477.patch, 15477v2.patch, 
> 15477v3.patch, 15477v3.patch, 15477v4.patch
>
>
> When we read from HDFS, we overread to pick up the next blocks header.
> Doing this saves a seek as we move through the hfile; we save having to
> do an explicit seek just to read the block header every time we need to
> read the body.  We used to read in the next header as part of the
> current blocks buffer. This buffer was then what got persisted to
> blockcache; so we were over-persisting wrtiting out our block plus the
> next blocks' header (overpersisting 33 bytes). Parse of HFileBlock
> complicated by this extra tail. Fix.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15389) Write out multiple files when compaction

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-15389:
--
Attachment: (was: HBASE-15389-v1-0.98.patch)

> Write out multiple files when compaction
> 
>
> Key: HBASE-15389
> URL: https://issues.apache.org/jira/browse/HBASE-15389
> Project: HBase
>  Issue Type: Sub-task
>  Components: Compaction
>Affects Versions: 2.0.0, 0.98.19, 1.4.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 0.98.19, 1.4.0
>
> Attachments: HBASE-15389-0.98.patch, HBASE-15389-branch-1.patch, 
> HBASE-15389-uc.patch, HBASE-15389-v1.patch, HBASE-15389-v10.patch, 
> HBASE-15389-v11.patch, HBASE-15389-v12.patch, HBASE-15389-v13.patch, 
> HBASE-15389-v2.patch, HBASE-15389-v3.patch, HBASE-15389-v4.patch, 
> HBASE-15389-v5.patch, HBASE-15389-v6.patch, HBASE-15389-v7.patch, 
> HBASE-15389-v8.patch, HBASE-15389-v9.patch, HBASE-15389.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15477) Do not save 'next block header' when we cache hfileblocks

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205835#comment-15205835
 ] 

stack commented on HBASE-15477:
---

bq. when we have some scans that involves scanning from contiguous blocks.

... and some blocks come from cache and some from HDFS... yeah, we keep the 
next-blocks length around and save it in cache so we avoid the need to seek the 
next block header before we can read the next block body.

Yeah, we retain this 'optimization' after all. It is not removed. It is 
documented on how it provides benefit (Just don't know how much).

> Do not save 'next block header' when we cache hfileblocks
> -
>
> Key: HBASE-15477
> URL: https://issues.apache.org/jira/browse/HBASE-15477
> Project: HBase
>  Issue Type: Sub-task
>  Components: BlockCache, Performance
>Reporter: stack
>Assignee: stack
> Attachments: 15366v4.patch, 15477.patch, 15477v2.patch, 
> 15477v3.patch, 15477v3.patch, 15477v4.patch
>
>
> When we read from HDFS, we overread to pick up the next blocks header.
> Doing this saves a seek as we move through the hfile; we save having to
> do an explicit seek just to read the block header every time we need to
> read the body.  We used to read in the next header as part of the
> current blocks buffer. This buffer was then what got persisted to
> blockcache; so we were over-persisting wrtiting out our block plus the
> next blocks' header (overpersisting 33 bytes). Parse of HFileBlock
> complicated by this extra tail. Fix.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15477) Do not save 'next block header' when we cache hfileblocks

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205827#comment-15205827
 ] 

stack commented on HBASE-15477:
---

bq. So we will NOT do "Do not save 'next block header' when we cache 
hfileblocks"?

We don't save the 'header' but we do save the length of the next block which is 
the important bit of info to have (for then we can read the next block in one 
seek... the next block's header comes in as part of this seek and will be 
sitting in the appropriate place in the buffer -- no need to do the 
back-filling header tricks).

> Do not save 'next block header' when we cache hfileblocks
> -
>
> Key: HBASE-15477
> URL: https://issues.apache.org/jira/browse/HBASE-15477
> Project: HBase
>  Issue Type: Sub-task
>  Components: BlockCache, Performance
>Reporter: stack
>Assignee: stack
> Attachments: 15366v4.patch, 15477.patch, 15477v2.patch, 
> 15477v3.patch, 15477v3.patch, 15477v4.patch
>
>
> When we read from HDFS, we overread to pick up the next blocks header.
> Doing this saves a seek as we move through the hfile; we save having to
> do an explicit seek just to read the block header every time we need to
> read the body.  We used to read in the next header as part of the
> current blocks buffer. This buffer was then what got persisted to
> blockcache; so we were over-persisting wrtiting out our block plus the
> next blocks' header (overpersisting 33 bytes). Parse of HFileBlock
> complicated by this extra tail. Fix.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15513) hbase.hregion.memstore.chunkpool.maxsize is 0.0 by default

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205824#comment-15205824
 ] 

Vladimir Rodionov commented on HBASE-15513:
---

[~saint@gmail.com]
{quote}
I should have added, I wonder what happens when you enable it?
{quote}

Good improvements in Old and Young GC times. It reduced Old GC events by 50% 
and time by 60%. See parent ticket. This MUST be enabled by default, I think.

The problem with disabling this pool - all new chunks (2MB each) are allocated 
in tenured space. Because we do not reuse them, only Old Gen GC will clean the 
space and Old GC can be very expensive sometimes. It affects Young GC as well. 



> hbase.hregion.memstore.chunkpool.maxsize is 0.0 by default
> --
>
> Key: HBASE-15513
> URL: https://issues.apache.org/jira/browse/HBASE-15513
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
>
> That results in excessive MemStoreLAB chunk allocations because we can not 
> reuse them. Not sure, why it has been disabled, by default. May be the code 
> has not been tested well?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15392) Single Cell Get reads two HFileBlocks

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-15392:
--
Attachment: HBASE-15265-v8.patch

Address [~lhofhansl] last comment

> Single Cell Get reads two HFileBlocks
> -
>
> Key: HBASE-15392
> URL: https://issues.apache.org/jira/browse/HBASE-15392
> Project: HBase
>  Issue Type: Sub-task
>  Components: BucketCache
>Reporter: stack
>Assignee: stack
> Attachments: 15392-0.98-looksee.txt, 15392.wip.patch, 
> 15392v2.wip.patch, 15392v3.wip.patch, 15392v4.patch, 15392v5.patch, 
> 15392v6.patch, 15392v7 (1).patch, 15392v7.patch, 15392v7.patch, 
> 15392v7.patch, 15392v7.patch, 15392v7.patch, HBASE-15265-v8.patch, 
> HBASE-15392_suggest.patch, gc.png, gc.png, io.png, no_optimize.patch, 
> no_optimize.patch, reads.png, reads.png, two_seeks.txt
>
>
> As found by Daniel "SystemTap" Pol, a simple Get results in our reading two 
> HFileBlocks, the one that contains the wanted Cell, and the block that 
> follows.
> Here is a bit of custom logging that logs a stack trace on each HFileBlock 
> read so you can see the call stack responsible:
> {code}
> 2016-03-03 22:20:30,191 INFO  
> [B.defaultRpcServer.handler=20,queue=2,port=16020] regionserver.StoreScanner: 
> START LOOP
> 2016-03-03 22:20:30,192 INFO  
> [B.defaultRpcServer.handler=20,queue=2,port=16020] regionserver.StoreScanner: 
> QCODE SEEK_NEXT_COL
> 2016-03-03 22:20:30,192 INFO  
> [B.defaultRpcServer.handler=20,queue=2,port=16020] hfile.HFileBlockIndex: 
> STARTED WHILE
> 2016-03-03 22:20:30,192 INFO  
> [B.defaultRpcServer.handler=20,queue=2,port=16020] hfile.CombinedBlockCache: 
> OUT OF L2
> 2016-03-03 22:20:30,192 TRACE 
> [B.defaultRpcServer.handler=20,queue=2,port=16020] bucket.BucketCache: Read 
> offset=31409152, len=2103
> 2016-03-03 22:20:30,192 TRACE 
> [B.defaultRpcServer.handler=20,queue=2,port=16020] bucket.FileIOEngine: 
> offset=31409152, length=2103
> 2016-03-03 22:20:30,193 TRACE 
> [B.defaultRpcServer.handler=20,queue=2,port=16020] hfile.HFileReaderImpl: 
> From Cache [blockType=DATA, fileOffset=2055421, headerSize=33, 
> onDiskSizeWithoutHeader=2024, uncompressedSizeWithoutHeader=2020, 
> prevBlockOffset=2053364, isUseHBaseChecksum=true, checksumType=CRC32C, 
> bytesPerChecksum=16384, onDiskDataSizeWithHeader=2053, 
> getOnDiskSizeWithHeader=2057, totalChecksumBytes=4, isUnpacked=true, 
> buf=[org.apache.hadoop.hbase.nio.SingleByteBuff@e19fbd54], 
> dataBeginsWith=\x00\x00\x00)\x00\x00\x01`\x00\x16user995139035672819231, 
> fileContext=[usesHBaseChecksum=true, checksumType=CRC32C, 
> bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true, 
> includesTags=false, compressAlgo=NONE, compressTags=false, 
> cryptoContext=[cipher=NONE keyHash=NONE]]]
> 2016-03-03 22:20:30,193 TRACE 
> [B.defaultRpcServer.handler=20,queue=2,port=16020] hfile.HFileReaderImpl: 
> Cache hit return [blockType=DATA, fileOffset=2055421, headerSize=33, 
> onDiskSizeWithoutHeader=2024, uncompressedSizeWithoutHeader=2020, 
> prevBlockOffset=2053364, isUseHBaseChecksum=true, checksumType=CRC32C, 
> bytesPerChecksum=16384, onDiskDataSizeWithHeader=2053, 
> getOnDiskSizeWithHeader=2057, totalChecksumBytes=4, isUnpacked=true, 
> buf=[org.apache.hadoop.hbase.nio.SingleByteBuff@e19fbd54], 
> dataBeginsWith=\x00\x00\x00)\x00\x00\x01`\x00\x16user995139035672819231, 
> fileContext=[usesHBaseChecksum=true, checksumType=CRC32C, 
> bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true, 
> includesTags=false, compressAlgo=NONE, compressTags=false, 
> cryptoContext=[cipher=NONE keyHash=NONE]]]
> java.lang.Throwable
> at 
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl.readBlock(HFileReaderImpl.java:1515)
> at 
> org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$CellBasedKeyBlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:324)
> at 
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.seekTo(HFileReaderImpl.java:831)
> at 
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.reseekTo(HFileReaderImpl.java:812)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:288)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:198)
> at 
> org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:54)
> at 
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:321)
> at 
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:279)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:806)
> at 
> 

[jira] [Commented] (HBASE-15392) Single Cell Get reads two HFileBlocks

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205821#comment-15205821
 ] 

stack commented on HBASE-15392:
---

You mean doing this?

{code}
diff --git 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
index 9cef059..66f846a 100644
--- 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
+++ 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
@@ -715,7 +715,7 @@ public class StoreScanner extends 
NonReversedNonLazyKeyValueScanner
 {
   Cell nextIndexedKey = getNextIndexedKey();
   if (nextIndexedKey != null && nextIndexedKey != 
KeyValueScanner.NO_NEXT_INDEXED_KEY
-  && matcher.compareKeyForNextColumn(nextIndexedKey, cell) >= 0) {
+  && matcher.compareKeyForNextColumn(nextIndexedKey, cell) > 0) {
 return qcode == MatchCode.SEEK_NEXT_COL ? MatchCode.SKIP : 
MatchCode.INCLUDE;
   }
   break;
@@ -730,7 +730,7 @@ public class StoreScanner extends 
NonReversedNonLazyKeyValueScanner
   if (!this.scan.isGetScan()) {
 Cell nextIndexedKey = getNextIndexedKey();
 if (nextIndexedKey != null && nextIndexedKey != 
KeyValueScanner.NO_NEXT_INDEXED_KEY
-&& matcher.compareKeyForNextRow(nextIndexedKey, cell) >= 0) {
+&& matcher.compareKeyForNextRow(nextIndexedKey, cell) > 0) {
   return qcode == MatchCode.SEEK_NEXT_ROW ? MatchCode.SKIP : 
MatchCode.INCLUDE;
 }
   }
{code}


> Single Cell Get reads two HFileBlocks
> -
>
> Key: HBASE-15392
> URL: https://issues.apache.org/jira/browse/HBASE-15392
> Project: HBase
>  Issue Type: Sub-task
>  Components: BucketCache
>Reporter: stack
>Assignee: stack
> Attachments: 15392-0.98-looksee.txt, 15392.wip.patch, 
> 15392v2.wip.patch, 15392v3.wip.patch, 15392v4.patch, 15392v5.patch, 
> 15392v6.patch, 15392v7 (1).patch, 15392v7.patch, 15392v7.patch, 
> 15392v7.patch, 15392v7.patch, 15392v7.patch, HBASE-15392_suggest.patch, 
> gc.png, gc.png, io.png, no_optimize.patch, no_optimize.patch, reads.png, 
> reads.png, two_seeks.txt
>
>
> As found by Daniel "SystemTap" Pol, a simple Get results in our reading two 
> HFileBlocks, the one that contains the wanted Cell, and the block that 
> follows.
> Here is a bit of custom logging that logs a stack trace on each HFileBlock 
> read so you can see the call stack responsible:
> {code}
> 2016-03-03 22:20:30,191 INFO  
> [B.defaultRpcServer.handler=20,queue=2,port=16020] regionserver.StoreScanner: 
> START LOOP
> 2016-03-03 22:20:30,192 INFO  
> [B.defaultRpcServer.handler=20,queue=2,port=16020] regionserver.StoreScanner: 
> QCODE SEEK_NEXT_COL
> 2016-03-03 22:20:30,192 INFO  
> [B.defaultRpcServer.handler=20,queue=2,port=16020] hfile.HFileBlockIndex: 
> STARTED WHILE
> 2016-03-03 22:20:30,192 INFO  
> [B.defaultRpcServer.handler=20,queue=2,port=16020] hfile.CombinedBlockCache: 
> OUT OF L2
> 2016-03-03 22:20:30,192 TRACE 
> [B.defaultRpcServer.handler=20,queue=2,port=16020] bucket.BucketCache: Read 
> offset=31409152, len=2103
> 2016-03-03 22:20:30,192 TRACE 
> [B.defaultRpcServer.handler=20,queue=2,port=16020] bucket.FileIOEngine: 
> offset=31409152, length=2103
> 2016-03-03 22:20:30,193 TRACE 
> [B.defaultRpcServer.handler=20,queue=2,port=16020] hfile.HFileReaderImpl: 
> From Cache [blockType=DATA, fileOffset=2055421, headerSize=33, 
> onDiskSizeWithoutHeader=2024, uncompressedSizeWithoutHeader=2020, 
> prevBlockOffset=2053364, isUseHBaseChecksum=true, checksumType=CRC32C, 
> bytesPerChecksum=16384, onDiskDataSizeWithHeader=2053, 
> getOnDiskSizeWithHeader=2057, totalChecksumBytes=4, isUnpacked=true, 
> buf=[org.apache.hadoop.hbase.nio.SingleByteBuff@e19fbd54], 
> dataBeginsWith=\x00\x00\x00)\x00\x00\x01`\x00\x16user995139035672819231, 
> fileContext=[usesHBaseChecksum=true, checksumType=CRC32C, 
> bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true, 
> includesTags=false, compressAlgo=NONE, compressTags=false, 
> cryptoContext=[cipher=NONE keyHash=NONE]]]
> 2016-03-03 22:20:30,193 TRACE 
> [B.defaultRpcServer.handler=20,queue=2,port=16020] hfile.HFileReaderImpl: 
> Cache hit return [blockType=DATA, fileOffset=2055421, headerSize=33, 
> onDiskSizeWithoutHeader=2024, uncompressedSizeWithoutHeader=2020, 
> prevBlockOffset=2053364, isUseHBaseChecksum=true, checksumType=CRC32C, 
> bytesPerChecksum=16384, onDiskDataSizeWithHeader=2053, 
> getOnDiskSizeWithHeader=2057, totalChecksumBytes=4, isUnpacked=true, 
> buf=[org.apache.hadoop.hbase.nio.SingleByteBuff@e19fbd54], 
> dataBeginsWith=\x00\x00\x00)\x00\x00\x01`\x00\x16user995139035672819231, 
> fileContext=[usesHBaseChecksum=true, checksumType=CRC32C, 
> 

[jira] [Commented] (HBASE-15492) Memory usage optimizations (Write path)

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205819#comment-15205819
 ] 

Vladimir Rodionov commented on HBASE-15492:
---

Forgot to mention ...

protobuf RPC serialization/deserialization is the major heap polluter now on 
both: client and server ( >> 50%). 

> Memory usage optimizations (Write path)
> ---
>
> Key: HBASE-15492
> URL: https://issues.apache.org/jira/browse/HBASE-15492
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: 5Patches.jfr, baseline.jfr, pre-HBASE-15479.jfr
>
>
> This is master ticket for all memory optimization tasks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15512) Avoid cell allocation on Store.add(Cell)

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205816#comment-15205816
 ] 

Vladimir Rodionov commented on HBASE-15512:
---

{quote}
We need to copy in order to get rid of the BB from the RpcServer or you need to 
ensure that the reference is done with this Cell.
{quote}

Will double check that, but test works OK. Making several copies of the same 
object is a bad habit.

> Avoid cell allocation on Store.add(Cell)
> 
>
> Key: HBASE-15512
> URL: https://issues.apache.org/jira/browse/HBASE-15512
> Project: HBase
>  Issue Type: Improvement
>  Components: Performance
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-15512-v1.patch
>
>
> AbstractMemStore.add(Cell) clones the cell. No reason for that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15493) Default ArrayList size may not be optimal for Mutation

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-15493:
--
Status: Patch Available  (was: Open)

> Default ArrayList size may not be optimal for Mutation
> --
>
> Key: HBASE-15493
> URL: https://issues.apache.org/jira/browse/HBASE-15493
> Project: HBase
>  Issue Type: Improvement
>  Components: Client, regionserver
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-15493-v1.patch, HBASE-15493-v2.patch
>
>
> {code}
>   List getCellList(byte[] family) {
> List list = this.familyMap.get(family);
> if (list == null) {
>   list = new ArrayList();
> }
> return list;
>   }
> {code}
> Creates list of size 10, this is up to 80 bytes per column family in mutation 
> object. 
> Suggested:
> {code}
>   List getCellList(byte[] family) {
> List list = this.familyMap.get(family);
> if (list == null) {
>   list = new ArrayList(CELL_LIST_INITIAL_CAPACITY);
> }
> return list;
>   }
> {code}
> CELL_LIST_INITIAL_CAPACITY = 2 in the patch, this is debatable. For mutation 
> where every CF has 1 cell, this gives decent reduction in memory allocation 
> rate in both client and server during write workload. ~2%, not a big number, 
> but as I said, already, memory optimization will include many small steps.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15493) Default ArrayList size may not be optimal for Mutation

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-15493:
--
Attachment: HBASE-15493-v2.patch

> Default ArrayList size may not be optimal for Mutation
> --
>
> Key: HBASE-15493
> URL: https://issues.apache.org/jira/browse/HBASE-15493
> Project: HBase
>  Issue Type: Improvement
>  Components: Client, regionserver
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-15493-v1.patch, HBASE-15493-v2.patch
>
>
> {code}
>   List getCellList(byte[] family) {
> List list = this.familyMap.get(family);
> if (list == null) {
>   list = new ArrayList();
> }
> return list;
>   }
> {code}
> Creates list of size 10, this is up to 80 bytes per column family in mutation 
> object. 
> Suggested:
> {code}
>   List getCellList(byte[] family) {
> List list = this.familyMap.get(family);
> if (list == null) {
>   list = new ArrayList(CELL_LIST_INITIAL_CAPACITY);
> }
> return list;
>   }
> {code}
> CELL_LIST_INITIAL_CAPACITY = 2 in the patch, this is debatable. For mutation 
> where every CF has 1 cell, this gives decent reduction in memory allocation 
> rate in both client and server during write workload. ~2%, not a big number, 
> but as I said, already, memory optimization will include many small steps.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15492) Memory usage optimizations (Write path)

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205810#comment-15205810
 ] 

Vladimir Rodionov commented on HBASE-15492:
---

Applied 4 patches on top of HBASE-15479: HBASE-15493, HBASE-15491, HBASE-15501, 
HBASE-15512. Set  *hbase.hregion.memstore.chunkpool.maxsize=0.4* (HBASE-15513). 
See attached 5patches.jfr.

Test run time 340 sec (down from 354 sec pre-HBASE-15479)
Total # of GC in young: 189 ( down from 360)
Total # of GC in old: 4 (down from 8)

Total time spent in GC: 6.7sec (down from 15 sec)

Averages and max GC times are also much better.

Total Allocation rate (combined TLAB + Old space) dropped from 1084MB/s to 
704MB/sec (in reality it is even less because it reflects increased 
throughput). 

> Memory usage optimizations (Write path)
> ---
>
> Key: HBASE-15492
> URL: https://issues.apache.org/jira/browse/HBASE-15492
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: 5Patches.jfr, baseline.jfr, pre-HBASE-15479.jfr
>
>
> This is master ticket for all memory optimization tasks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15492) Memory usage optimizations (Write path)

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-15492:
--
Attachment: 5Patches.jfr

> Memory usage optimizations (Write path)
> ---
>
> Key: HBASE-15492
> URL: https://issues.apache.org/jira/browse/HBASE-15492
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: 5Patches.jfr, baseline.jfr, pre-HBASE-15479.jfr
>
>
> This is master ticket for all memory optimization tasks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15265) Implement an asynchronous FSHLog

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-15265:
--
Attachment: HBASE-15265-v8.patch

See if this all passes now we undid the upgrade to surefire 2.19.1 which seems 
to be getting rid of the timeouts.

> Implement an asynchronous FSHLog
> 
>
> Key: HBASE-15265
> URL: https://issues.apache.org/jira/browse/HBASE-15265
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-15265-v1.patch, HBASE-15265-v2.patch, 
> HBASE-15265-v3.patch, HBASE-15265-v4.patch, HBASE-15265-v5.patch, 
> HBASE-15265-v6.patch, HBASE-15265-v7.patch, HBASE-15265-v8.patch, 
> HBASE-15265-v8.patch, HBASE-15265.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15265) Implement an asynchronous FSHLog

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205791#comment-15205791
 ] 

stack commented on HBASE-15265:
---

Sorry [~Apache9] My cluster got reimaged so got set back a little (all for the 
best though).

> Implement an asynchronous FSHLog
> 
>
> Key: HBASE-15265
> URL: https://issues.apache.org/jira/browse/HBASE-15265
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-15265-v1.patch, HBASE-15265-v2.patch, 
> HBASE-15265-v3.patch, HBASE-15265-v4.patch, HBASE-15265-v5.patch, 
> HBASE-15265-v6.patch, HBASE-15265-v7.patch, HBASE-15265-v8.patch, 
> HBASE-15265.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15514) Single (or small number) of Column Family(ies) in Mutation optimization

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205788#comment-15205788
 ] 

stack commented on HBASE-15514:
---

A start might be purging references to family Map from public API then we could 
start doing something else (CellBlocks!!!)

> Single (or small number) of Column Family(ies) in Mutation optimization
> ---
>
> Key: HBASE-15514
> URL: https://issues.apache.org/jira/browse/HBASE-15514
> Project: HBase
>  Issue Type: Improvement
>  Components: Performance
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>
> familyMap in Mutation class is TreeMap. Overhead of 
> TreeMap - 48 bytes, TreeMap$Entry - 40 bytes. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15513) hbase.hregion.memstore.chunkpool.maxsize is 0.0 by default

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205783#comment-15205783
 ] 

stack commented on HBASE-15513:
---

I should have added, I wonder what happens when you enable it?

> hbase.hregion.memstore.chunkpool.maxsize is 0.0 by default
> --
>
> Key: HBASE-15513
> URL: https://issues.apache.org/jira/browse/HBASE-15513
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
>
> That results in excessive MemStoreLAB chunk allocations because we can not 
> reuse them. Not sure, why it has been disabled, by default. May be the code 
> has not been tested well?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15509) Avoid copy of block data in HFileBlock$Writer.finishBlock

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205782#comment-15205782
 ] 

stack commented on HBASE-15509:
---

Sorry. Missed this comment of yours [~ram_krish]

> Avoid copy of block data in HFileBlock$Writer.finishBlock
> -
>
> Key: HBASE-15509
> URL: https://issues.apache.org/jira/browse/HBASE-15509
> Project: HBase
>  Issue Type: Improvement
>  Components: HFile
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>
> This calls ByteArrayOutputStream.toByteArray() which creates a copy of data 
> block. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15510) Reuse buffer inside RpcServer$Connection.readAndProcess

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205780#comment-15205780
 ] 

stack commented on HBASE-15510:
---

How is this different to HBASE-14490?

> Reuse buffer inside RpcServer$Connection.readAndProcess
> ---
>
> Key: HBASE-15510
> URL: https://issues.apache.org/jira/browse/HBASE-15510
> Project: HBase
>  Issue Type: Improvement
>  Components: IPC/RPC
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
>
> ByteBuffer is created on every call. 
> Server-side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15510) Reuse buffer inside RpcServer$Connection.readAndProcess

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205779#comment-15205779
 ] 

stack commented on HBASE-15510:
---

So, less GC but GC takes more time?

> Reuse buffer inside RpcServer$Connection.readAndProcess
> ---
>
> Key: HBASE-15510
> URL: https://issues.apache.org/jira/browse/HBASE-15510
> Project: HBase
>  Issue Type: Improvement
>  Components: IPC/RPC
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
>
> ByteBuffer is created on every call. 
> Server-side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15509) Avoid copy of block data in HFileBlock$Writer.finishBlock

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205778#comment-15205778
 ] 

stack commented on HBASE-15509:
---

It says in code:

  // This does an array copy, so it is safe to cache this byte array when 
cache-on-write.
  // Header is still the empty, 'dummy' header that is yet to be filled out.

So perhaps, if we are not doing cache-on-write, we don't need to allocate at 
all?

Could try putting in place the BoundedByteBufferPool here? Could then easily 
have the pool be offheap.

> Avoid copy of block data in HFileBlock$Writer.finishBlock
> -
>
> Key: HBASE-15509
> URL: https://issues.apache.org/jira/browse/HBASE-15509
> Project: HBase
>  Issue Type: Improvement
>  Components: HFile
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>
> This calls ByteArrayOutputStream.toByteArray() which creates a copy of data 
> block. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15513) hbase.hregion.memstore.chunkpool.maxsize is 0.0 by default

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205776#comment-15205776
 ] 

stack commented on HBASE-15513:
---

See https://issues.apache.org/jira/browse/HBASE-8163 It was intentionally off 
by default.

> hbase.hregion.memstore.chunkpool.maxsize is 0.0 by default
> --
>
> Key: HBASE-15513
> URL: https://issues.apache.org/jira/browse/HBASE-15513
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
>
> That results in excessive MemStoreLAB chunk allocations because we can not 
> reuse them. Not sure, why it has been disabled, by default. May be the code 
> has not been tested well?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15515) Improve LocalityBasedCandidateGenerator in Balancer

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-15515:
---
Status: Patch Available  (was: Open)

> Improve LocalityBasedCandidateGenerator in Balancer
> ---
>
> Key: HBASE-15515
> URL: https://issues.apache.org/jira/browse/HBASE-15515
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15515.patch
>
>
> There are some problems which need to fix.
> 1. LocalityBasedCandidateGenerator.getLowestLocalityRegionOnServer should 
> skip empty region.
> 2. When use LocalityBasedCandidateGenerator to generate Cluster.Action, it 
> should add random operation instead of pickLowestLocalityServer(cluster). 
> Because the search function may stuck here if it always generate the same 
> Cluster.Action.
> 3. getLeastLoadedTopServerForRegion should get least loaded server which have 
> better locality than current server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15510) Reuse buffer inside RpcServer$Connection.readAndProcess

[Elliott Clark (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205773#comment-15205773
 ] 

Elliott Clark commented on HBASE-15510:
---

Most of the times we've tried to share/reuse buffers on the heap things have 
been much slower on a newer jdk/gc.

> Reuse buffer inside RpcServer$Connection.readAndProcess
> ---
>
> Key: HBASE-15510
> URL: https://issues.apache.org/jira/browse/HBASE-15510
> Project: HBase
>  Issue Type: Improvement
>  Components: IPC/RPC
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
>
> ByteBuffer is created on every call. 
> Server-side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15325) ResultScanner allowing partial result will miss the rest of the row if the region is moved between two rpc requests

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205772#comment-15205772
 ] 

Hudson commented on HBASE-15325:


FAILURE: Integrated in HBase-1.4 #41 (See 
[https://builds.apache.org/job/HBase-1.4/41/])
HBASE-15325 ResultScanner allowing partial result will miss the rest of (tedyu: 
rev 1c5002660ba8e28e71e4a91c66d6a52ce7d4beb9)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/TestPartialResultsFromClientSide.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java


> ResultScanner allowing partial result will miss the rest of the row if the 
> region is moved between two rpc requests
> ---
>
> Key: HBASE-15325
> URL: https://issues.apache.org/jira/browse/HBASE-15325
> Project: HBase
>  Issue Type: Bug
>  Components: dataloss, Scanners
>Affects Versions: 1.2.0, 1.1.3
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.1.4, 1.4.0, 1.2.2
>
> Attachments: 15325-test.txt, HBASE-15325-branch-1-v1.patch, 
> HBASE-15325-branch-1-v2.patch, HBASE-15325-branch-1.1-v1.patch, 
> HBASE-15325-branch-1.3-v1.patch, HBASE-15325-branch-1.3-v2.patch, 
> HBASE-15325-v1.txt, HBASE-15325-v10.patch, HBASE-15325-v11.patch, 
> HBASE-15325-v2.txt, HBASE-15325-v3.txt, HBASE-15325-v5.txt, 
> HBASE-15325-v6.1.txt, HBASE-15325-v6.2.txt, HBASE-15325-v6.3.txt, 
> HBASE-15325-v6.4.txt, HBASE-15325-v6.5.txt, HBASE-15325-v6.txt, 
> HBASE-15325-v7.patch, HBASE-15325-v8.patch, HBASE-15325-v9.patch
>
>
> HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for 
> one rpc request. And client can setAllowPartial or setBatch to get several 
> cells in a row instead of the whole row.
> However, the status of the scanner is saved on server and we need this to get 
> the next part if there is a partial result before. If we move the region to 
> another RS, client will get a NotServingRegionException and open a new 
> scanner to the new RS which will be regarded as a new scan from the end of 
> this row. So the rest cells of the row of last result will be missing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15512) Avoid cell allocation on Store.add(Cell)

[ramkrishna.s.vasudevan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205770#comment-15205770
 ] 

ramkrishna.s.vasudevan commented on HBASE-15512:


I agree with Stack here. We need to copy in order to get rid of the BB from the 
RpcServer or you need to ensure that the reference is done with this Cell. 

> Avoid cell allocation on Store.add(Cell)
> 
>
> Key: HBASE-15512
> URL: https://issues.apache.org/jira/browse/HBASE-15512
> Project: HBase
>  Issue Type: Improvement
>  Components: Performance
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-15512-v1.patch
>
>
> AbstractMemStore.add(Cell) clones the cell. No reason for that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15493) Default ArrayList size may not be optimal for Mutation

[ramkrishna.s.vasudevan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205765#comment-15205765
 ] 

ramkrishna.s.vasudevan commented on HBASE-15493:


I would say this could be of lower priority now. But the other JIRAs 
HBASE-15509 (during flushes and compactions) and HBASE-15506 may be real 
winners for us. 
Anoop's HBASE-15180 was a big gainer for us. 

> Default ArrayList size may not be optimal for Mutation
> --
>
> Key: HBASE-15493
> URL: https://issues.apache.org/jira/browse/HBASE-15493
> Project: HBase
>  Issue Type: Improvement
>  Components: Client, regionserver
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-15493-v1.patch
>
>
> {code}
>   List getCellList(byte[] family) {
> List list = this.familyMap.get(family);
> if (list == null) {
>   list = new ArrayList();
> }
> return list;
>   }
> {code}
> Creates list of size 10, this is up to 80 bytes per column family in mutation 
> object. 
> Suggested:
> {code}
>   List getCellList(byte[] family) {
> List list = this.familyMap.get(family);
> if (list == null) {
>   list = new ArrayList(CELL_LIST_INITIAL_CAPACITY);
> }
> return list;
>   }
> {code}
> CELL_LIST_INITIAL_CAPACITY = 2 in the patch, this is debatable. For mutation 
> where every CF has 1 cell, this gives decent reduction in memory allocation 
> rate in both client and server during write workload. ~2%, not a big number, 
> but as I said, already, memory optimization will include many small steps.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15510) Reuse buffer inside RpcServer$Connection.readAndProcess

[ramkrishna.s.vasudevan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205763#comment-15205763
 ] 

ramkrishna.s.vasudevan commented on HBASE-15510:


Yes. But reusing BBpool even offheap though is reducing the GC cycle it is 
adding to the GC time. We are in the process of analysing this part. 

> Reuse buffer inside RpcServer$Connection.readAndProcess
> ---
>
> Key: HBASE-15510
> URL: https://issues.apache.org/jira/browse/HBASE-15510
> Project: HBase
>  Issue Type: Improvement
>  Components: IPC/RPC
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
>
> ByteBuffer is created on every call. 
> Server-side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15509) Avoid copy of block data in HFileBlock$Writer.finishBlock

[ramkrishna.s.vasudevan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205761#comment-15205761
 ] 

ramkrishna.s.vasudevan commented on HBASE-15509:


All these are in our TODO list while doing our POC on write path. 
This is the biggest garbage generator. Planning to add an byte[] pool?

> Avoid copy of block data in HFileBlock$Writer.finishBlock
> -
>
> Key: HBASE-15509
> URL: https://issues.apache.org/jira/browse/HBASE-15509
> Project: HBase
>  Issue Type: Improvement
>  Components: HFile
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>
> This calls ByteArrayOutputStream.toByteArray() which creates a copy of data 
> block. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15514) Single (or small number) of Column Family(ies) in Mutation optimization

[ramkrishna.s.vasudevan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205758#comment-15205758
 ] 

ramkrishna.s.vasudevan commented on HBASE-15514:


Better to convert all these as subtasks. 
So familyMap has to do a clone() every time. That is generating lot of garbage. 
But I don't think we can avoid that copy if a Map with byte[] is used. 

> Single (or small number) of Column Family(ies) in Mutation optimization
> ---
>
> Key: HBASE-15514
> URL: https://issues.apache.org/jira/browse/HBASE-15514
> Project: HBase
>  Issue Type: Improvement
>  Components: Performance
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>
> familyMap in Mutation class is TreeMap. Overhead of 
> TreeMap - 48 bytes, TreeMap$Entry - 40 bytes. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15493) Default ArrayList size may not be optimal for Mutation

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205757#comment-15205757
 ] 

stack commented on HBASE-15493:
---

bq. ... do not want to spend a lot of time on this easy fix

I don't either. The 'fix' is for timeseries usecase at expense of others; e.g. 
loadings with many columns.. now they do an extra few resizings... they slow 
down. What about above suggestion of histogram of sizings... over a period of 
time. We need it all over the hbase codebase. There is a related attempt in the 
BoundedByteBufferPool.

> Default ArrayList size may not be optimal for Mutation
> --
>
> Key: HBASE-15493
> URL: https://issues.apache.org/jira/browse/HBASE-15493
> Project: HBase
>  Issue Type: Improvement
>  Components: Client, regionserver
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-15493-v1.patch
>
>
> {code}
>   List getCellList(byte[] family) {
> List list = this.familyMap.get(family);
> if (list == null) {
>   list = new ArrayList();
> }
> return list;
>   }
> {code}
> Creates list of size 10, this is up to 80 bytes per column family in mutation 
> object. 
> Suggested:
> {code}
>   List getCellList(byte[] family) {
> List list = this.familyMap.get(family);
> if (list == null) {
>   list = new ArrayList(CELL_LIST_INITIAL_CAPACITY);
> }
> return list;
>   }
> {code}
> CELL_LIST_INITIAL_CAPACITY = 2 in the patch, this is debatable. For mutation 
> where every CF has 1 cell, this gives decent reduction in memory allocation 
> rate in both client and server during write workload. ~2%, not a big number, 
> but as I said, already, memory optimization will include many small steps.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15411) Rewrite backup with Procedure V2

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205755#comment-15205755
 ] 

Ted Yu commented on HBASE-15411:


Created https://reviews.apache.org/r/45149/

Let's continue review there.

> Rewrite backup with Procedure V2
> 
>
> Key: HBASE-15411
> URL: https://issues.apache.org/jira/browse/HBASE-15411
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15411-v1.txt, 15411-v11.txt, 15411-v12.txt, 
> 15411-v13.txt, 15411-v14.txt, 15411-v15.txt, 15411-v16.txt, 15411-v18.txt, 
> 15411-v3.txt, 15411-v5.txt, 15411-v6.txt, 15411-v7.txt, 15411-v9.txt, 
> FullTableBackupProcedure.java
>
>
> Currently full / incremental backup is driven by BackupHandler (see call() 
> method for flow).
> This issue is to rewrite the flow using Procedure V2.
> States (enum) for full / incremental backup would be introduced in 
> Backup.proto which correspond to the steps performed in BackupHandler#call().
> executeFromState() would pace the backup based on the current state.
> serializeStateData() / deserializeStateData() would be used to persist state 
> into procedure WAL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15512) Avoid cell allocation on Store.add(Cell)

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205747#comment-15205747
 ] 

stack commented on HBASE-15512:
---

How is this safe? We pass the KV out of the Segment method. We sure that the 
previous reference is done with the KV? We won't be overwriting a KV that is in 
use?

> Avoid cell allocation on Store.add(Cell)
> 
>
> Key: HBASE-15512
> URL: https://issues.apache.org/jira/browse/HBASE-15512
> Project: HBase
>  Issue Type: Improvement
>  Components: Performance
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-15512-v1.patch
>
>
> AbstractMemStore.add(Cell) clones the cell. No reason for that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15411) Rewrite backup with Procedure V2

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205744#comment-15205744
 ] 

Ted Yu commented on HBASE-15411:


bq. This connection and Admin is leaked

In the next patch, try-with-resources is used to prevent the leak.

> Rewrite backup with Procedure V2
> 
>
> Key: HBASE-15411
> URL: https://issues.apache.org/jira/browse/HBASE-15411
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15411-v1.txt, 15411-v11.txt, 15411-v12.txt, 
> 15411-v13.txt, 15411-v14.txt, 15411-v15.txt, 15411-v16.txt, 15411-v18.txt, 
> 15411-v3.txt, 15411-v5.txt, 15411-v6.txt, 15411-v7.txt, 15411-v9.txt, 
> FullTableBackupProcedure.java
>
>
> Currently full / incremental backup is driven by BackupHandler (see call() 
> method for flow).
> This issue is to rewrite the flow using Procedure V2.
> States (enum) for full / incremental backup would be introduced in 
> Backup.proto which correspond to the steps performed in BackupHandler#call().
> executeFromState() would pace the backup based on the current state.
> serializeStateData() / deserializeStateData() would be used to persist state 
> into procedure WAL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HBASE-15411) Rewrite backup with Procedure V2

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205738#comment-15205738
 ] 

Ted Yu edited comment on HBASE-15411 at 3/22/16 4:04 AM:
-

bq. Lets gather these arguments in a client-public class like BackupRequest

I went through existing methods in Admin.java but didn't see such pattern. 
Since BackupRequest is not protobuf'ed, when new field is introduced, 
compilation would break anyway.

bq. Where is the removed code, for example BackupHandler etc?

Yes, the code in FullTableBackupProcedure.java came from BackupHandler.
I would suggest keeping FullTableBackupProcedure and BackupHandler close for 
phase I so that Vlad's work in phase 2 can be easily picked up.

bq. This workers is being carried all the way from the client.

The workers parameter would be used once we come to rebasing Vlad's phase 2 
work.

bq. we want exclusive lock on the table, not shared, no?

After offline discussion with Stephen, we chose shared locks.


was (Author: yuzhih...@gmail.com):
bq. Lets gather these arguments in a client-public class like BackupRequest

I went through existing methods in Admin.java but didn't see such pattern. 
Since BackupRequest is not protobuf'ed, when new field is introduced, 
compilation would break anyway.

bq. Where is the removed code, for example BackupHandler etc?

Yes, the code in FullTableBackupProcedure.java came from BackupHandler.
I would suggest keep FullTableBackupProcedure and BackupHandler close for phase 
I so that Vlad's work in phase 2 can be easily picked up.

bq. This workers is being carried all the way from the client.

The workers parameter would be used once we come to rebasing Vlad's phase 2 
work.

bq. we want exclusive lock on the table, not shared, no?

After offline discussion with Stephen, we chose shared locks.

> Rewrite backup with Procedure V2
> 
>
> Key: HBASE-15411
> URL: https://issues.apache.org/jira/browse/HBASE-15411
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15411-v1.txt, 15411-v11.txt, 15411-v12.txt, 
> 15411-v13.txt, 15411-v14.txt, 15411-v15.txt, 15411-v16.txt, 15411-v18.txt, 
> 15411-v3.txt, 15411-v5.txt, 15411-v6.txt, 15411-v7.txt, 15411-v9.txt, 
> FullTableBackupProcedure.java
>
>
> Currently full / incremental backup is driven by BackupHandler (see call() 
> method for flow).
> This issue is to rewrite the flow using Procedure V2.
> States (enum) for full / incremental backup would be introduced in 
> Backup.proto which correspond to the steps performed in BackupHandler#call().
> executeFromState() would pace the backup based on the current state.
> serializeStateData() / deserializeStateData() would be used to persist state 
> into procedure WAL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15496) Throw RowTooBigException only for user scan/get

[Guanghao Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205741#comment-15205741
 ] 

Guanghao Zhang commented on HBASE-15496:


For huge keyvalue case, compaction may produces OOME server side.

> Throw RowTooBigException only for user scan/get
> ---
>
> Key: HBASE-15496
> URL: https://issues.apache.org/jira/browse/HBASE-15496
> Project: HBase
>  Issue Type: Improvement
>  Components: Scanners
>Reporter: Guanghao Zhang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15496.patch
>
>
> When config hbase.table.max.rowsize, RowTooBigException may be thrown by 
> StoreScanner. But region flush/compact should catch it or throw it only for 
> user scan.
> Exceptions:
> org.apache.hadoop.hbase.regionserver.RowTooBigException: Max row size 
> allowed: 10485760, but row is bigger than that
>   at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:355)
>   at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:276)
>   at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:238)
>   at 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:403)
>   at 
> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:95)
>   at 
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:131)
>   at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1211)
>   at org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1952)
>   at org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1774)
> or 
> org.apache.hadoop.hbase.regionserver.RowTooBigException: Max row size 
> allowed: 10485760, but the row is bigger than that.
>   at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:576)
>   at 
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:132)
>   at 
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
>   at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:880)
>   at 
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2155)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2454)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2193)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2162)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2053)
>   at org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:1979)
> at 
> org.apache.hadoop.hbase.regionserver.TestRowTooBig.testScannersSeekOnFewLargeCells(TestRowTooBig.java:101)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15411) Rewrite backup with Procedure V2

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205738#comment-15205738
 ] 

Ted Yu commented on HBASE-15411:


bq. Lets gather these arguments in a client-public class like BackupRequest

I went through existing methods in Admin.java but didn't see such pattern. 
Since BackupRequest is not protobuf'ed, when new field is introduced, 
compilation would break anyway.

bq. Where is the removed code, for example BackupHandler etc?

Yes, the code in FullTableBackupProcedure.java came from BackupHandler.
I would suggest keep FullTableBackupProcedure and BackupHandler close for phase 
I so that Vlad's work in phase 2 can be easily picked up.

bq. This workers is being carried all the way from the client.

The workers parameter would be used once we come to rebasing Vlad's phase 2 
work.

bq. we want exclusive lock on the table, not shared, no?

After offline discussion with Stephen, we chose shared locks.

> Rewrite backup with Procedure V2
> 
>
> Key: HBASE-15411
> URL: https://issues.apache.org/jira/browse/HBASE-15411
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15411-v1.txt, 15411-v11.txt, 15411-v12.txt, 
> 15411-v13.txt, 15411-v14.txt, 15411-v15.txt, 15411-v16.txt, 15411-v18.txt, 
> 15411-v3.txt, 15411-v5.txt, 15411-v6.txt, 15411-v7.txt, 15411-v9.txt, 
> FullTableBackupProcedure.java
>
>
> Currently full / incremental backup is driven by BackupHandler (see call() 
> method for flow).
> This issue is to rewrite the flow using Procedure V2.
> States (enum) for full / incremental backup would be introduced in 
> Backup.proto which correspond to the steps performed in BackupHandler#call().
> executeFromState() would pace the backup based on the current state.
> serializeStateData() / deserializeStateData() would be used to persist state 
> into procedure WAL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15479) No more garbage or beware of autoboxing

[Yu Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205737#comment-15205737
 ] 

Yu Li commented on HBASE-15479:
---

Good to know, thanks. Will also check the performance in local if time allows.

btw, it seems no _parent ticket_ for this one, maybe lack of some link here?

> No more garbage or beware of autoboxing
> ---
>
> Key: HBASE-15479
> URL: https://issues.apache.org/jira/browse/HBASE-15479
> Project: HBase
>  Issue Type: Bug
>  Components: Client
>Affects Versions: 2.0.0, 1.1.0, 1.2.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0, 1.3.0, 1.2.1, 0.98.19, 1.4.0, 1.1.5
>
> Attachments: HBASE-15479-v1.patch, HBASE-15479-v2.patch
>
>
> Quick journey with JMC in a profile mode revealed very interesting and 
> unexpected heap polluter on a client side. Patch will shortly follow. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15488) Add ACL for setting split merge switch

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-15488:
---
Attachment: (was: HBASE-15488.v3.patch)

> Add ACL for setting split merge switch
> --
>
> Key: HBASE-15488
> URL: https://issues.apache.org/jira/browse/HBASE-15488
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-15488-branch-1.v1.patch, 
> HBASE-15488-branch-1.v2.patch, HBASE-15488.v1.patch, HBASE-15488.v2.patch, 
> HBASE-15488.v3.patch
>
>
> Currently there is no access control for the split merge switch setter in 
> MasterRpcServices.
> This JIRA adds necessary coprocessor hook along with enforcing permission 
> check in AccessController through the new hook.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15488) Add ACL for setting split merge switch

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-15488:
---
Attachment: HBASE-15488.v3.patch

> Add ACL for setting split merge switch
> --
>
> Key: HBASE-15488
> URL: https://issues.apache.org/jira/browse/HBASE-15488
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-15488-branch-1.v1.patch, 
> HBASE-15488-branch-1.v2.patch, HBASE-15488.v1.patch, HBASE-15488.v2.patch, 
> HBASE-15488.v3.patch
>
>
> Currently there is no access control for the split merge switch setter in 
> MasterRpcServices.
> This JIRA adds necessary coprocessor hook along with enforcing permission 
> check in AccessController through the new hook.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15300) Upgrade to zookeeper 3.4.8

[Yu Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205733#comment-15205733
 ] 

Yu Li commented on HBASE-15300:
---

+1, ZOOKEEPER-706 is indeed an important fix

Also checked the timed out cases 
(TestMasterFailoverWithProcedures/TestRegionMergeTransactionOnCluster) in local 
with zookeeper version updated to 3.4.8, and confirmed all could pass

> Upgrade to zookeeper 3.4.8
> --
>
> Key: HBASE-15300
> URL: https://issues.apache.org/jira/browse/HBASE-15300
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
>  Labels: zookeeper
> Fix For: 2.0.0
>
> Attachments: HBASE-15300.v1.patch, HBASE-15300.v2.patch, 
> HBASE-15300.v2.patch, HBASE-15300.v2.patch, HBASE-1533.v1.patch
>
>
> zookeeper 3.4.8 has been released.
> Among the fixes the following are desirable:
> ZOOKEEPER-706 large numbers of watches can cause session re-establishment to 
> fail 
> ZOOKEEPER-1797 PurgeTxnLog may delete data logs during roll
> This issue upgrades zookeeper dependency to 3.4.8



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15265) Implement an asynchronous FSHLog

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-15265:
--
Attachment: HBASE-15265-v8.patch

Sorry I have uploaded a wrong version...

> Implement an asynchronous FSHLog
> 
>
> Key: HBASE-15265
> URL: https://issues.apache.org/jira/browse/HBASE-15265
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-15265-v1.patch, HBASE-15265-v2.patch, 
> HBASE-15265-v3.patch, HBASE-15265-v4.patch, HBASE-15265-v5.patch, 
> HBASE-15265-v6.patch, HBASE-15265-v7.patch, HBASE-15265-v8.patch, 
> HBASE-15265.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15265) Implement an asynchronous FSHLog

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-15265:
--
Attachment: (was: HBASE-15265-v8.patch)

> Implement an asynchronous FSHLog
> 
>
> Key: HBASE-15265
> URL: https://issues.apache.org/jira/browse/HBASE-15265
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-15265-v1.patch, HBASE-15265-v2.patch, 
> HBASE-15265-v3.patch, HBASE-15265-v4.patch, HBASE-15265-v5.patch, 
> HBASE-15265-v6.patch, HBASE-15265-v7.patch, HBASE-15265.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15265) Implement an asynchronous FSHLog

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-15265:
--
Attachment: HBASE-15265-v8.patch

Rebase.

[~stack] Any progress? :)

> Implement an asynchronous FSHLog
> 
>
> Key: HBASE-15265
> URL: https://issues.apache.org/jira/browse/HBASE-15265
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-15265-v1.patch, HBASE-15265-v2.patch, 
> HBASE-15265-v3.patch, HBASE-15265-v4.patch, HBASE-15265-v5.patch, 
> HBASE-15265-v6.patch, HBASE-15265-v7.patch, HBASE-15265-v8.patch, 
> HBASE-15265.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205694#comment-15205694
 ] 

Vladimir Rodionov commented on HBASE-14123:
---

{quote}
Can you please revert those changes?
{quote}

[~enis], What changes? Can you past the code?

> HBase Backup/Restore Phase 2
> 
>
> Key: HBASE-14123
> URL: https://issues.apache.org/jira/browse/HBASE-14123
> Project: HBase
>  Issue Type: Umbrella
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-14123-v1.patch, HBASE-14123-v10.patch, 
> HBASE-14123-v11.patch, HBASE-14123-v12.patch, HBASE-14123-v2.patch, 
> HBASE-14123-v3.patch, HBASE-14123-v4.patch, HBASE-14123-v5.patch, 
> HBASE-14123-v6.patch, HBASE-14123-v7.patch, HBASE-14123-v9.patch
>
>
> Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15460) Fix infer issues in hbase-common

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205690#comment-15205690
 ] 

Hudson commented on HBASE-15460:


SUCCESS: Integrated in HBase-1.3-IT #573 (See 
[https://builds.apache.org/job/HBase-1.3-IT/573/])
HBASE-15460 Fix infer issues in hbase-common (eclark: rev 
295e5cbcf038bbd86cd5dbc05ae652f09b5c5bdf)
* 
hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/util/JVM.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/util/Base64.java
* 
hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java
* 
hbase-common/src/main/java/org/apache/hadoop/hbase/util/test/LoadTestKVGenerator.java


> Fix infer issues in hbase-common
> 
>
> Key: HBASE-15460
> URL: https://issues.apache.org/jira/browse/HBASE-15460
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15460-v1.patch, HBASE-15460.patch, 
> hbase-common.infer
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205693#comment-15205693
 ] 

Vladimir Rodionov commented on HBASE-14123:
---

[~enis] let us finish (commit) Phase 1 first. 

> HBase Backup/Restore Phase 2
> 
>
> Key: HBASE-14123
> URL: https://issues.apache.org/jira/browse/HBASE-14123
> Project: HBase
>  Issue Type: Umbrella
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-14123-v1.patch, HBASE-14123-v10.patch, 
> HBASE-14123-v11.patch, HBASE-14123-v12.patch, HBASE-14123-v2.patch, 
> HBASE-14123-v3.patch, HBASE-14123-v4.patch, HBASE-14123-v5.patch, 
> HBASE-14123-v6.patch, HBASE-14123-v7.patch, HBASE-14123-v9.patch
>
>
> Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15325) ResultScanner allowing partial result will miss the rest of the row if the region is moved between two rpc requests

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205691#comment-15205691
 ] 

Hudson commented on HBASE-15325:


SUCCESS: Integrated in HBase-1.3-IT #573 (See 
[https://builds.apache.org/job/HBase-1.3-IT/573/])
HBASE-15325 ResultScanner allowing partial result will miss the rest of (tedyu: 
rev 1c5002660ba8e28e71e4a91c66d6a52ce7d4beb9)
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/TestPartialResultsFromClientSide.java


> ResultScanner allowing partial result will miss the rest of the row if the 
> region is moved between two rpc requests
> ---
>
> Key: HBASE-15325
> URL: https://issues.apache.org/jira/browse/HBASE-15325
> Project: HBase
>  Issue Type: Bug
>  Components: dataloss, Scanners
>Affects Versions: 1.2.0, 1.1.3
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.1.4, 1.4.0, 1.2.2
>
> Attachments: 15325-test.txt, HBASE-15325-branch-1-v1.patch, 
> HBASE-15325-branch-1-v2.patch, HBASE-15325-branch-1.1-v1.patch, 
> HBASE-15325-branch-1.3-v1.patch, HBASE-15325-branch-1.3-v2.patch, 
> HBASE-15325-v1.txt, HBASE-15325-v10.patch, HBASE-15325-v11.patch, 
> HBASE-15325-v2.txt, HBASE-15325-v3.txt, HBASE-15325-v5.txt, 
> HBASE-15325-v6.1.txt, HBASE-15325-v6.2.txt, HBASE-15325-v6.3.txt, 
> HBASE-15325-v6.4.txt, HBASE-15325-v6.5.txt, HBASE-15325-v6.txt, 
> HBASE-15325-v7.patch, HBASE-15325-v8.patch, HBASE-15325-v9.patch
>
>
> HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for 
> one rpc request. And client can setAllowPartial or setBatch to get several 
> cells in a row instead of the whole row.
> However, the status of the scanner is saved on server and we need this to get 
> the next part if there is a partial result before. If we move the region to 
> another RS, client will get a NotServingRegionException and open a new 
> scanner to the new RS which will be regarded as a new scan from the end of 
> this row. So the rest cells of the row of last result will be missing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15389) Write out multiple files when compaction

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205689#comment-15205689
 ] 

Duo Zhang commented on HBASE-15389:
---

And let's commit this first before HBASE-15400? Then I could start working on 
the archive issue and the refactoring work of {{Compactor}} which I promised to 
[~enis]...


> Write out multiple files when compaction
> 
>
> Key: HBASE-15389
> URL: https://issues.apache.org/jira/browse/HBASE-15389
> Project: HBase
>  Issue Type: Sub-task
>  Components: Compaction
>Affects Versions: 2.0.0, 0.98.19, 1.4.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 0.98.19, 1.4.0
>
> Attachments: HBASE-15389-0.98.patch, HBASE-15389-branch-1.patch, 
> HBASE-15389-uc.patch, HBASE-15389-v1-0.98.patch, HBASE-15389-v1.patch, 
> HBASE-15389-v10.patch, HBASE-15389-v11.patch, HBASE-15389-v12.patch, 
> HBASE-15389-v13.patch, HBASE-15389-v2.patch, HBASE-15389-v3.patch, 
> HBASE-15389-v4.patch, HBASE-15389-v5.patch, HBASE-15389-v6.patch, 
> HBASE-15389-v7.patch, HBASE-15389-v8.patch, HBASE-15389-v9.patch, 
> HBASE-15389.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15464) Flush / Compaction metrics revisited

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205692#comment-15205692
 ] 

Hudson commented on HBASE-15464:


SUCCESS: Integrated in HBase-1.3-IT #573 (See 
[https://builds.apache.org/job/HBase-1.3-IT/573/])
HBASE-15464 Flush / Compaction metrics revisited (enis: rev 
249e37f83c32a8231824d4e84f298e3b344e158a)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFlushContext.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreEngine.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java
* 
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java
* 
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionContext.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/flush/FlushTableSubprocedure.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/MockRegionServerServices.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStripeStoreEngine.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreEngine.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java


> Flush / Compaction metrics revisited
> 
>
> Key: HBASE-15464
> URL: https://issues.apache.org/jira/browse/HBASE-15464
> Project: HBase
>  Issue Type: Sub-task
>  Components: metrics
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: hbase-15464_v1.patch, hbase-15464_v2.patch, 
> hbase-15464_v3.patch, hbase-15464_v3.patch, hbase-15464_v4.patch
>
>
> We can add a couple of metrics related to flushes and compactions: 
>  - flush memstore and output file size histogram: This will allow seeing 
> whether we are flushing too early due to memory pressure, too many regions, 
> etc. Tracking flush memstore size vs output file size is useful in 
> understanding the block encoding compression benefits. 
>  - total flushed output bytes: This will allow to monitor the IO / throughput 
> from flushers. You can use this to set num flushers, flush throttle, etc. 
>  - smallCompactionQueueLength / large...: This is tracked, but not emitted 
> anymore due to a bug. 
>  - compaction time histogram: similar to flush time histogram, how long 
> compactions are taking. 
>  - compaction input num files / output num files histogram: How many files on 
> average we are compacting. Stripe compaction / date tiered compaction can use 
> the num output files metric. 
>  - compaction input / output data sizes histogram: How much data on average 
> we are compacting. 
>  - compaction input / output total bytes: Measure compaction IO / throughput. 
> measure write amplification, enables to set compaction throttle. 
>  - Breakdown for above for major compactions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15236) Inconsistent cell reads over multiple bulk-loaded HFiles

[Appy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-15236:
-
Attachment: HBASE-15236-master-v6.patch

same patch for re-building.

> Inconsistent cell reads over multiple bulk-loaded HFiles
> 
>
> Key: HBASE-15236
> URL: https://issues.apache.org/jira/browse/HBASE-15236
> Project: HBase
>  Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-15236-master-v2.patch, 
> HBASE-15236-master-v3.patch, HBASE-15236-master-v4.patch, 
> HBASE-15236-master-v5.patch, HBASE-15236-master-v6.patch, HBASE-15236.patch, 
> TestWithSingleHRegion.java, tables_data.zip
>
>
>  If there are two bulkloaded hfiles in a region with same seqID, same 
> timestamps and duplicate keys*, get and scan may return different values for 
> a key. Not sure how this would happen, but one of our customer uploaded a 
> dataset with 2 files in a single region and both having same bulk load 
> timestamp. These files are small ~50M (I couldn't find any setting for max 
> file size that could lead to 2 files). The range of keys in two hfiles are 
> overlapping to some extent, but not fully (so the two files are because of 
> region merge).
> In such a case, depending on file sizes (used in 
> StoreFile.Comparators.SEQ_ID), we may get different values for the same cell 
> (say "r", "cf:50") depending on what we call: get "r" "cf:50" or get "r" 
> "cf:".
> To replicate this, i ran ImportTsv twice with 2 different csv files but same 
> timestamp (-Dimporttsv.timestamp=1). Collected two resulting hfiles in a 
> single directory and loaded with LoadIncrementalHFiles. Here are the commands.
> {noformat}
> //CSV files should be in hdfs:///user/hbase/tmp
> sudo -u hbase hbase org.apache.hadoop.hbase.mapreduce.ImportTsv  
> -Dimporttsv.columns=HBASE_ROW_KEY,c:1,c:2,c:3,c:4,c:5,c:6 
> -Dimporttsv.separator='|' -Dimporttsv.timestamp=1 
> -Dimporttsv.bulk.output=hdfs:///user/hbase/1 t tmp
> {noformat}
> tables_data.zip contains three tables: t, t2, and t3.
> To load these tables and test with them, use the attached 
> TestWithSingleHRegion.java. (Change ROOT_DIR and TABLE_NAME variables 
> appropriately)
> Hfiles for table 't':
> 1) 07922ac90208445883b2ddc89c424c89 : length = 1094: KVs = (c:5, 55) (c:6, 66)
> 2) 143137fa4fa84625b4e8d1d84a494f08 : length = 1192: KVs = (c:1, 1) (c:2, 2) 
> (c:3, 3) (c:4, 4) (c:5, 5) (c:6, 6)
> Get returns 5  where as Scan returns 55.
> On the other hand if I make the first hfile, the one with values 55 and 66 
> bigger in size than second hfile, then:
> Get returns 55  where as Scan returns 55.
> This can be seen in table 't2'.
> Weird things:
> 1. Get consistently returns values from larger hfile.
> 2. Scan consistently returns 55.
> Reason:
> Explanation 1: We add StoreFileScanners to heap by iterating over list of 
> store files which is kept sorted in DefaultStoreFileManger using 
> StoreFile.Comparators.SEQ_ID. It sorts the file first by increasing seq id, 
> then by decreasing filesize. So our larger files sizes are always in the 
> start.
> Explanation 2: In KeyValueHeap.next(), if both current and this.heap.peek() 
> scanners (type is StoreFileScanner in this case) point to KVs which compare 
> as equal, we put back the current scanner into heap, and call pollRealKV() to 
> get new one. While inserting, KVScannerComparator is used which compares the 
> two (one already in heap and the one being inserted) as equal and inserts it 
> after the existing one. \[1\]
> Say s1 is scanner over first hfile and s2 over second.
> 1. We scan with s2 to get values 1, 2, 3, 4
> 2. see that both scanners have c:5 as next key, put current scanner s2 back 
> in heap, take s1 out
> 4. return 55, advance s1 to key c:6
> 5. compare c:6 to c:5, put back s2 in heap since it has larger key, take out 
> s1
> 6. ignore it's value (there is code for that too), advance s2 to c:6
> Go back to step 2 with both scanners having c:6 as next key
> This is wrong because if we add a key between c:4 and c:5 to first hfile, say 
> (c:44, foo)  which makes s1 as the 'current' scanner when we hit step 2, then 
> we'll get the values - 1, 2, 3, 4, foo, 5, 6.
> (try table t3)
> Fix:
> Assign priority ids to StoreFileScanners when inserting them into heap, 
> latest hfiles get higher priority, and use them in KVComparator instead of 
> just seq id.
> \[1\] 
> [PriorityQueue.siftUp()|http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/util/PriorityQueue.java#586]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15488) Add ACL for setting split merge switch

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-15488:
---
Attachment: HBASE-15488.v3.patch

Patch v3 fixes a bug in MasterCoprocessorHost#preSetSplitOrMergeEnabled()

> Add ACL for setting split merge switch
> --
>
> Key: HBASE-15488
> URL: https://issues.apache.org/jira/browse/HBASE-15488
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-15488-branch-1.v1.patch, 
> HBASE-15488-branch-1.v2.patch, HBASE-15488.v1.patch, HBASE-15488.v2.patch, 
> HBASE-15488.v3.patch
>
>
> Currently there is no access control for the split merge switch setter in 
> MasterRpcServices.
> This JIRA adds necessary coprocessor hook along with enforcing permission 
> check in AccessController through the new hook.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15515) Improve LocalityBasedCandidateGenerator in Balancer

[Guanghao Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-15515:
---
Attachment: HBASE-15515.patch

> Improve LocalityBasedCandidateGenerator in Balancer
> ---
>
> Key: HBASE-15515
> URL: https://issues.apache.org/jira/browse/HBASE-15515
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15515.patch
>
>
> There are some problems which need to fix.
> 1. LocalityBasedCandidateGenerator.getLowestLocalityRegionOnServer should 
> skip empty region.
> 2. When use LocalityBasedCandidateGenerator to generate Cluster.Action, it 
> should add random operation instead of pickLowestLocalityServer(cluster). 
> Because the search function may stuck here if it always generate the same 
> Cluster.Action.
> 3. getLeastLoadedTopServerForRegion should get least loaded server which have 
> better locality than current server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15515) Improve LocalityBasedCandidateGenerator in Balancer

[Guanghao Zhang (JIRA)]

Guanghao Zhang created HBASE-15515:
--

 Summary: Improve LocalityBasedCandidateGenerator in Balancer
 Key: HBASE-15515
 URL: https://issues.apache.org/jira/browse/HBASE-15515
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0, 1.3.0
Reporter: Guanghao Zhang
Assignee: Guanghao Zhang
Priority: Minor
 Fix For: 2.0.0


There are some problems which need to fix.
1. LocalityBasedCandidateGenerator.getLowestLocalityRegionOnServer should skip 
empty region.
2. When use LocalityBasedCandidateGenerator to generate Cluster.Action, it 
should add random operation instead of pickLowestLocalityServer(cluster). 
Because the search function may stuck here if it always generate the same 
Cluster.Action.
3. getLeastLoadedTopServerForRegion should get least loaded server which have 
better locality than current server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15411) Rewrite backup with Procedure V2

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205671#comment-15205671
 ] 

Enis Soztutar commented on HBASE-15411:
---

To create an RB for reviewing this, I have committed the patch for HBASE-14030 
(v42) to the apache branch HBASE-7912. We can use this branch at Apache as a 
working branch for now (meaning that we can use this for rebases and committing 
patches before full-review cycle. 

Review board is here: https://reviews.apache.org/r/45146/diff/1#index_header. 
I'm not sure whether I can change ownership of the RB for future patch 
iterations, if not, you can re-create another RB. 

It seems that if you do
{code}
git diff --full-index
{code} 
RB will automatically pick up the correct base for the patch. 

> Rewrite backup with Procedure V2
> 
>
> Key: HBASE-15411
> URL: https://issues.apache.org/jira/browse/HBASE-15411
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15411-v1.txt, 15411-v11.txt, 15411-v12.txt, 
> 15411-v13.txt, 15411-v14.txt, 15411-v15.txt, 15411-v16.txt, 15411-v18.txt, 
> 15411-v3.txt, 15411-v5.txt, 15411-v6.txt, 15411-v7.txt, 15411-v9.txt, 
> FullTableBackupProcedure.java
>
>
> Currently full / incremental backup is driven by BackupHandler (see call() 
> method for flow).
> This issue is to rewrite the flow using Procedure V2.
> States (enum) for full / incremental backup would be introduced in 
> Backup.proto which correspond to the steps performed in BackupHandler#call().
> executeFromState() would pace the backup based on the current state.
> serializeStateData() / deserializeStateData() would be used to persist state 
> into procedure WAL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15495) Connection leak in FanOutOneBlockAsyncDFSOutputHelper

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-15495:
--
Attachment: HBASE-15495-v2.patch

Addressing the hadoop compatibility issue.

> Connection leak in FanOutOneBlockAsyncDFSOutputHelper
> -
>
> Key: HBASE-15495
> URL: https://issues.apache.org/jira/browse/HBASE-15495
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Attachments: HBASE-15495-v1.patch, HBASE-15495-v2.patch, 
> HBASE-15495.patch
>
>
> Happens when connecting to datanode failed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15325) ResultScanner allowing partial result will miss the rest of the row if the region is moved between two rpc requests

[Phil Yang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205669#comment-15205669
 ] 

Phil Yang commented on HBASE-15325:
---

HBASE-15325-branch-1.3-v2.patch pass all tests, HBASE-15325-branch-1-v2.patch 
fail some tests but they are unrelated and can pass locally.

> ResultScanner allowing partial result will miss the rest of the row if the 
> region is moved between two rpc requests
> ---
>
> Key: HBASE-15325
> URL: https://issues.apache.org/jira/browse/HBASE-15325
> Project: HBase
>  Issue Type: Bug
>  Components: dataloss, Scanners
>Affects Versions: 1.2.0, 1.1.3
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.1.4, 1.4.0, 1.2.2
>
> Attachments: 15325-test.txt, HBASE-15325-branch-1-v1.patch, 
> HBASE-15325-branch-1-v2.patch, HBASE-15325-branch-1.1-v1.patch, 
> HBASE-15325-branch-1.3-v1.patch, HBASE-15325-branch-1.3-v2.patch, 
> HBASE-15325-v1.txt, HBASE-15325-v10.patch, HBASE-15325-v11.patch, 
> HBASE-15325-v2.txt, HBASE-15325-v3.txt, HBASE-15325-v5.txt, 
> HBASE-15325-v6.1.txt, HBASE-15325-v6.2.txt, HBASE-15325-v6.3.txt, 
> HBASE-15325-v6.4.txt, HBASE-15325-v6.5.txt, HBASE-15325-v6.txt, 
> HBASE-15325-v7.patch, HBASE-15325-v8.patch, HBASE-15325-v9.patch
>
>
> HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for 
> one rpc request. And client can setAllowPartial or setBatch to get several 
> cells in a row instead of the whole row.
> However, the status of the scanner is saved on server and we need this to get 
> the next part if there is a partial result before. If we move the region to 
> another RS, client will get a NotServingRegionException and open a new 
> scanner to the new RS which will be regarded as a new scan from the end of 
> this row. So the rest cells of the row of last result will be missing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15389) Write out multiple files when compaction

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-15389:
--
Affects Version/s: (was: 1.3.0)
   1.4.0
Fix Version/s: (was: 1.3.0)
   1.4.0

> Write out multiple files when compaction
> 
>
> Key: HBASE-15389
> URL: https://issues.apache.org/jira/browse/HBASE-15389
> Project: HBase
>  Issue Type: Sub-task
>  Components: Compaction
>Affects Versions: 2.0.0, 0.98.19, 1.4.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 0.98.19, 1.4.0
>
> Attachments: HBASE-15389-0.98.patch, HBASE-15389-branch-1.patch, 
> HBASE-15389-uc.patch, HBASE-15389-v1-0.98.patch, HBASE-15389-v1.patch, 
> HBASE-15389-v10.patch, HBASE-15389-v11.patch, HBASE-15389-v12.patch, 
> HBASE-15389-v13.patch, HBASE-15389-v2.patch, HBASE-15389-v3.patch, 
> HBASE-15389-v4.patch, HBASE-15389-v5.patch, HBASE-15389-v6.patch, 
> HBASE-15389-v7.patch, HBASE-15389-v8.patch, HBASE-15389-v9.patch, 
> HBASE-15389.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15287) org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with binary row key inputs

[Matt Warhaftig (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205663#comment-15205663
 ] 

Matt Warhaftig commented on HBASE-15287:


Hi [~enis], Randy's description of the issue (and fix) are correct and would be 
a quick patch.  I had been investigating the larger question of whether other 
HBase MapReduce functions should get the same change.

I grabbed the ticket figuring on a 3/30 ETA for patch submission but am ok 
giving it up to someone else.

> org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with 
> binary row key inputs
> 
>
> Key: HBASE-15287
> URL: https://issues.apache.org/jira/browse/HBASE-15287
> Project: HBase
>  Issue Type: Bug
>  Components: mapreduce, util
>Affects Versions: 1.1.1
>Reporter: Randy Hu
>Assignee: Matt Warhaftig
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> org.apache.hadoop.hbase.mapreduce.RowCounter takes optional start/end key as 
> inputs (-range option). It would work only when the string representation of 
> value is identical to the string. When row key is binary,  the string 
> representation of the value would look like this: "\x00\x01", which would be 
> incorrect interpreted as 8 char string in the current implementation:
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/RowCounter.java
> To fix that, we need change how the value is converted from command line 
> inputs:
> Change 
>   scan.setStartRow(Bytes.toBytes(startKey));
> to
>   scan.setStartRow(Bytes.toBytesBinary(startKey));
> Do the same conversion to end key as well.
> The issue was discovered when the utility was used to calcualte row 
> distribution on regions from table with binary row keys. The hbase:meta 
> contains the start key of each region in format of above example. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15488) Add ACL for setting split merge switch

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205660#comment-15205660
 ] 

Hadoop QA commented on HBASE-15488:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 32s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s 
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m 
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s 
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 29s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 29s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s 
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 2s 
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 6m 34s 
{color} | {color:red} hbase-server: patch generated 5 new + 121 unchanged - 0 
fixed = 126 total (was 121) {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 27s 
{color} | {color:red} Patch causes 14 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m 45s 
{color} | {color:red} Patch causes 14 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m 1s 
{color} | {color:red} Patch causes 14 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 9m 20s 
{color} | {color:red} Patch causes 14 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 11m 38s 
{color} | {color:red} Patch causes 14 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 14m 0s 
{color} | {color:red} Patch causes 14 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 16m 19s 
{color} | {color:red} Patch causes 14 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 18m 35s 
{color} | {color:red} Patch causes 14 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 20m 44s 
{color} | {color:red} Patch causes 14 errors with Hadoop v2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 17s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 17s 
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 144m 34s 
{color} | 

[jira] [Updated] (HBASE-15514) Single (or small number) of Column Family(ies) in Mutation optimization

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-15514:
--
Summary: Single (or small number) of Column Family(ies) in Mutation 
optimization  (was: Single (or small number) of Column Family in Mutation 
optimization)

> Single (or small number) of Column Family(ies) in Mutation optimization
> ---
>
> Key: HBASE-15514
> URL: https://issues.apache.org/jira/browse/HBASE-15514
> Project: HBase
>  Issue Type: Improvement
>  Components: Performance
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>
> familyMap in Mutation class is TreeMap. Overhead of 
> TreeMap - 48 bytes, TreeMap$Entry - 40 bytes. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15514) Single (or small number) of Column Family in Mutation optimization

[Vladimir Rodionov (JIRA)]

Vladimir Rodionov created HBASE-15514:
-

 Summary: Single (or small number) of Column Family in Mutation 
optimization
 Key: HBASE-15514
 URL: https://issues.apache.org/jira/browse/HBASE-15514
 Project: HBase
  Issue Type: Improvement
  Components: Performance
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov


familyMap in Mutation class is TreeMap. Overhead of TreeMap 
- 48 bytes, TreeMap$Entry - 40 bytes. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15512) Avoid cell allocation on Store.add(Cell)

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-15512:
--
Status: Patch Available  (was: Open)

> Avoid cell allocation on Store.add(Cell)
> 
>
> Key: HBASE-15512
> URL: https://issues.apache.org/jira/browse/HBASE-15512
> Project: HBase
>  Issue Type: Improvement
>  Components: Performance
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-15512-v1.patch
>
>
> AbstractMemStore.add(Cell) clones the cell. No reason for that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15389) Write out multiple files when compaction

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205653#comment-15205653
 ] 

Duo Zhang commented on HBASE-15389:
---

So let's set the fix version to 1.4.0? Then we have plenty of time for testing.

> Write out multiple files when compaction
> 
>
> Key: HBASE-15389
> URL: https://issues.apache.org/jira/browse/HBASE-15389
> Project: HBase
>  Issue Type: Sub-task
>  Components: Compaction
>Affects Versions: 2.0.0, 1.3.0, 0.98.19
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 0.98.19
>
> Attachments: HBASE-15389-0.98.patch, HBASE-15389-branch-1.patch, 
> HBASE-15389-uc.patch, HBASE-15389-v1-0.98.patch, HBASE-15389-v1.patch, 
> HBASE-15389-v10.patch, HBASE-15389-v11.patch, HBASE-15389-v12.patch, 
> HBASE-15389-v13.patch, HBASE-15389-v2.patch, HBASE-15389-v3.patch, 
> HBASE-15389-v4.patch, HBASE-15389-v5.patch, HBASE-15389-v6.patch, 
> HBASE-15389-v7.patch, HBASE-15389-v8.patch, HBASE-15389-v9.patch, 
> HBASE-15389.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15512) Avoid cell allocation on Store.add(Cell)

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-15512:
--
Attachment: HBASE-15512-v1.patch

Patch v1. Old idea of having method in KeyValue for reusing object. It works.
~2% reduction in allocation rate on server-side.

> Avoid cell allocation on Store.add(Cell)
> 
>
> Key: HBASE-15512
> URL: https://issues.apache.org/jira/browse/HBASE-15512
> Project: HBase
>  Issue Type: Improvement
>  Components: Performance
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-15512-v1.patch
>
>
> AbstractMemStore.add(Cell) clones the cell. No reason for that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15389) Write out multiple files when compaction

[Clara Xiong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205646#comment-15205646
 ] 

Clara Xiong commented on HBASE-15389:
-

I don't have any more code review concerns. Can we couple the release of 
HBASE-15389 and HBASE-15400? I am porting this to our branch to test in 
production this week. And I'd like to get more thorough review on HBASE-15400 
from people familiar with Ratio-based compaction policy.

> Write out multiple files when compaction
> 
>
> Key: HBASE-15389
> URL: https://issues.apache.org/jira/browse/HBASE-15389
> Project: HBase
>  Issue Type: Sub-task
>  Components: Compaction
>Affects Versions: 2.0.0, 1.3.0, 0.98.19
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 0.98.19
>
> Attachments: HBASE-15389-0.98.patch, HBASE-15389-branch-1.patch, 
> HBASE-15389-uc.patch, HBASE-15389-v1-0.98.patch, HBASE-15389-v1.patch, 
> HBASE-15389-v10.patch, HBASE-15389-v11.patch, HBASE-15389-v12.patch, 
> HBASE-15389-v13.patch, HBASE-15389-v2.patch, HBASE-15389-v3.patch, 
> HBASE-15389-v4.patch, HBASE-15389-v5.patch, HBASE-15389-v6.patch, 
> HBASE-15389-v7.patch, HBASE-15389-v8.patch, HBASE-15389-v9.patch, 
> HBASE-15389.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14030) HBase Backup/Restore Phase 1

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-14030:
---
Attachment: HBASE-14030.v43.patch

Patch v43 adopts the naming Enis suggested.

> HBase Backup/Restore Phase 1
> 
>
> Key: HBASE-14030
> URL: https://issues.apache.org/jira/browse/HBASE-14030
> Project: HBase
>  Issue Type: Umbrella
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-14030-v0.patch, HBASE-14030-v1.patch, 
> HBASE-14030-v10.patch, HBASE-14030-v11.patch, HBASE-14030-v12.patch, 
> HBASE-14030-v13.patch, HBASE-14030-v14.patch, HBASE-14030-v15.patch, 
> HBASE-14030-v17.patch, HBASE-14030-v18.patch, HBASE-14030-v2.patch, 
> HBASE-14030-v20.patch, HBASE-14030-v21.patch, HBASE-14030-v22.patch, 
> HBASE-14030-v23.patch, HBASE-14030-v24.patch, HBASE-14030-v25.patch, 
> HBASE-14030-v26.patch, HBASE-14030-v27.patch, HBASE-14030-v28.patch, 
> HBASE-14030-v3.patch, HBASE-14030-v30.patch, HBASE-14030-v35.patch, 
> HBASE-14030-v37.patch, HBASE-14030-v4.patch, HBASE-14030-v5.patch, 
> HBASE-14030-v6.patch, HBASE-14030-v7.patch, HBASE-14030-v8.patch, 
> HBASE-14030.v38.patch, HBASE-14030.v39.patch, HBASE-14030.v40.patch, 
> HBASE-14030.v41.patch, HBASE-14030.v42.patch, HBASE-14030.v43.patch, 
> hbase-14030_v36.patch
>
>
> This is the umbrella ticket for Backup/Restore Phase 1. See HBASE-7912 design 
> doc for the phase description.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15287) org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with binary row key inputs

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205642#comment-15205642
 ] 

Enis Soztutar commented on HBASE-15287:
---

You have a patch? 

> org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with 
> binary row key inputs
> 
>
> Key: HBASE-15287
> URL: https://issues.apache.org/jira/browse/HBASE-15287
> Project: HBase
>  Issue Type: Bug
>  Components: mapreduce, util
>Affects Versions: 1.1.1
>Reporter: Randy Hu
>Assignee: Matt Warhaftig
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> org.apache.hadoop.hbase.mapreduce.RowCounter takes optional start/end key as 
> inputs (-range option). It would work only when the string representation of 
> value is identical to the string. When row key is binary,  the string 
> representation of the value would look like this: "\x00\x01", which would be 
> incorrect interpreted as 8 char string in the current implementation:
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/RowCounter.java
> To fix that, we need change how the value is converted from command line 
> inputs:
> Change 
>   scan.setStartRow(Bytes.toBytes(startKey));
> to
>   scan.setStartRow(Bytes.toBytesBinary(startKey));
> Do the same conversion to end key as well.
> The issue was discovered when the utility was used to calcualte row 
> distribution on regions from table with binary row keys. The hbase:meta 
> contains the start key of each region in format of above example. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HBASE-15287) org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with binary row key inputs

[Matt Warhaftig (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Warhaftig reassigned HBASE-15287:
--

Assignee: Matt Warhaftig

> org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with 
> binary row key inputs
> 
>
> Key: HBASE-15287
> URL: https://issues.apache.org/jira/browse/HBASE-15287
> Project: HBase
>  Issue Type: Bug
>  Components: mapreduce, util
>Affects Versions: 1.1.1
>Reporter: Randy Hu
>Assignee: Matt Warhaftig
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> org.apache.hadoop.hbase.mapreduce.RowCounter takes optional start/end key as 
> inputs (-range option). It would work only when the string representation of 
> value is identical to the string. When row key is binary,  the string 
> representation of the value would look like this: "\x00\x01", which would be 
> incorrect interpreted as 8 char string in the current implementation:
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/RowCounter.java
> To fix that, we need change how the value is converted from command line 
> inputs:
> Change 
>   scan.setStartRow(Bytes.toBytes(startKey));
> to
>   scan.setStartRow(Bytes.toBytesBinary(startKey));
> Do the same conversion to end key as well.
> The issue was discovered when the utility was used to calcualte row 
> distribution on regions from table with binary row keys. The hbase:meta 
> contains the start key of each region in format of above example. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14147) REST Support for Namespaces

[Matt Warhaftig (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205636#comment-15205636
 ] 

Matt Warhaftig commented on HBASE-14147:


Hi [~misty],  It looks like specifying {{-H "Content-Type: text/xml"}} results 
in a 400 Bad Request if the curl statement does not also include the -data 
argument.

To create a namespace with no namespace properties specified you can do:
 {noformat}curl -vi -X POST -H "Accept: text/xml" 
"http://localhost:20550/namespaces/namespaceName"{noformat}

Or to create a namespace with property key1=value1 specified via xml you can do:
{noformat}curl -vi -X POST -H "Accept: text/xml" -H "Content-Type: text/xml" -d 
"key1value1"
 "http://localhost:20550/namespaces/namespaceName"{noformat}

> REST Support for Namespaces
> ---
>
> Key: HBASE-14147
> URL: https://issues.apache.org/jira/browse/HBASE-14147
> Project: HBase
>  Issue Type: Sub-task
>  Components: REST
>Affects Versions: 1.1.1
>Reporter: Rick Kellogg
>Assignee: Matt Warhaftig
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.15
>
> Attachments: SuccessfulLocalBuild.txt, hbase-14147-v1.patch, 
> hbase-14147-v1.patch, hbase-14147-v1.patch
>
>
> Expand REST services to include addition features:
> * Create namespace
> * Alter namespace
> * Describe namespace
> * Drop namespace
> * List tables in a specific namespace
> * List all namespaces.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15504) Fix Balancer in 1.3 not moving regions off overloaded regionserver

[Guanghao Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205628#comment-15205628
 ] 

Guanghao Zhang commented on HBASE-15504:


And when use StochasticLoadBalancer in our test cluster, we found some other 
problems which need to fix.
1. LocalityBasedCandidateGenerator.getLowestLocalityRegionOnServer should skip 
region which have nothing.
2. When use LocalityBasedCandidateGenerator to generate Cluster.Action, it 
should add random operation instead of pickLowestLocalityServer(cluster). 
Because the search function may stuck here if it always generate the same 
Cluster.Action.
3. getLeastLoadedTopServerForRegion(int region) should get least loaded server 
which have better locality than current server.

> Fix Balancer in 1.3 not moving regions off overloaded regionserver
> --
>
> Key: HBASE-15504
> URL: https://issues.apache.org/jira/browse/HBASE-15504
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Elliott Clark
> Fix For: 1.3.0
>
>
> We pushed 1.3 to a couple of clusters. In some cases the regions were 
> assigned VERY un-evenly and the regions would not move after that.
> We ended up with one rs getting thousands of regions and most servers getting 
> 0. Running balancer would do nothing. The balancer would say that it couldn't 
> find a solution with less than the current cost.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15360) Fix flaky TestSimpleRpcScheduler

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205614#comment-15205614
 ] 

Ted Yu commented on HBASE-15360:


The failed test was introduced by HBASE-15136 which was mentioned in the 
description of the JIRA.

> Fix flaky TestSimpleRpcScheduler
> 
>
> Key: HBASE-15360
> URL: https://issues.apache.org/jira/browse/HBASE-15360
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Mikhail Antonov
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15360.patch
>
>
> There were several flaky tests added there as part of HBASE-15306 and likely 
> HBASE-15136.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15495) Connection leak in FanOutOneBlockAsyncDFSOutputHelper

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205610#comment-15205610
 ] 

Duo Zhang commented on HBASE-15495:
---

Let me see the hadoop compatibility issues...

> Connection leak in FanOutOneBlockAsyncDFSOutputHelper
> -
>
> Key: HBASE-15495
> URL: https://issues.apache.org/jira/browse/HBASE-15495
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Attachments: HBASE-15495-v1.patch, HBASE-15495.patch
>
>
> Happens when connecting to datanode failed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15389) Write out multiple files when compaction

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205608#comment-15205608
 ] 

Duo Zhang commented on HBASE-15389:
---

[~enis] [~davelatham] [~clarax98007] If no new concerns, let's commit this and 
focus on HBASE-15400?

Thanks.

> Write out multiple files when compaction
> 
>
> Key: HBASE-15389
> URL: https://issues.apache.org/jira/browse/HBASE-15389
> Project: HBase
>  Issue Type: Sub-task
>  Components: Compaction
>Affects Versions: 2.0.0, 1.3.0, 0.98.19
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 0.98.19
>
> Attachments: HBASE-15389-0.98.patch, HBASE-15389-branch-1.patch, 
> HBASE-15389-uc.patch, HBASE-15389-v1-0.98.patch, HBASE-15389-v1.patch, 
> HBASE-15389-v10.patch, HBASE-15389-v11.patch, HBASE-15389-v12.patch, 
> HBASE-15389-v13.patch, HBASE-15389-v2.patch, HBASE-15389-v3.patch, 
> HBASE-15389-v4.patch, HBASE-15389-v5.patch, HBASE-15389-v6.patch, 
> HBASE-15389-v7.patch, HBASE-15389-v8.patch, HBASE-15389-v9.patch, 
> HBASE-15389.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

[Jianwei Cui (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205607#comment-15205607
 ] 

Jianwei Cui commented on HBASE-15433:
-

Thanks for the review, Ted and Ashish.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15433-branch-1-v1.patch, 
> HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14030) HBase Backup/Restore Phase 1

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205604#comment-15205604
 ] 

Enis Soztutar commented on HBASE-14030:
---

bq. If you have further suggestion on how they can be merged, that would be 
great.
I did not take a close look at the code to see how and whether they can be 
merged or not. Can you please either do the analysis and merge or rename one of 
the classes to be BackupClientUtil or something. 

> HBase Backup/Restore Phase 1
> 
>
> Key: HBASE-14030
> URL: https://issues.apache.org/jira/browse/HBASE-14030
> Project: HBase
>  Issue Type: Umbrella
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-14030-v0.patch, HBASE-14030-v1.patch, 
> HBASE-14030-v10.patch, HBASE-14030-v11.patch, HBASE-14030-v12.patch, 
> HBASE-14030-v13.patch, HBASE-14030-v14.patch, HBASE-14030-v15.patch, 
> HBASE-14030-v17.patch, HBASE-14030-v18.patch, HBASE-14030-v2.patch, 
> HBASE-14030-v20.patch, HBASE-14030-v21.patch, HBASE-14030-v22.patch, 
> HBASE-14030-v23.patch, HBASE-14030-v24.patch, HBASE-14030-v25.patch, 
> HBASE-14030-v26.patch, HBASE-14030-v27.patch, HBASE-14030-v28.patch, 
> HBASE-14030-v3.patch, HBASE-14030-v30.patch, HBASE-14030-v35.patch, 
> HBASE-14030-v37.patch, HBASE-14030-v4.patch, HBASE-14030-v5.patch, 
> HBASE-14030-v6.patch, HBASE-14030-v7.patch, HBASE-14030-v8.patch, 
> HBASE-14030.v38.patch, HBASE-14030.v39.patch, HBASE-14030.v40.patch, 
> HBASE-14030.v41.patch, HBASE-14030.v42.patch, hbase-14030_v36.patch
>
>
> This is the umbrella ticket for Backup/Restore Phase 1. See HBASE-7912 design 
> doc for the phase description.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205600#comment-15205600
 ] 

Enis Soztutar commented on HBASE-14123:
---

bq. Describe in release notes that hfile validation is turned off.
This does not look right. 

Just skimming the patch, I noticed that this patch undoes some of my changed to 
the HBASE-14030, for example BackupSystemTable.java. Can you please revert 
those changes. 

In offline discussions, my big concern on the review side was HBASE-15443 and 
HBASE-15442 and similar issues that touch the layout / file formats. Is there a 
way we can extract these changes and push them to the HBASE-14030 patch? 

Do you have a write up that summarizes the backup layout, backup image and 
backup manifest contents? It is extremely important to get the backup file 
layout and structures right, so such a document will definitely help others get 
the overall picture. Also, we can see whether the new proposed layout plays 
nicely with this format or not. 

> HBase Backup/Restore Phase 2
> 
>
> Key: HBASE-14123
> URL: https://issues.apache.org/jira/browse/HBASE-14123
> Project: HBase
>  Issue Type: Umbrella
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-14123-v1.patch, HBASE-14123-v10.patch, 
> HBASE-14123-v11.patch, HBASE-14123-v12.patch, HBASE-14123-v2.patch, 
> HBASE-14123-v3.patch, HBASE-14123-v4.patch, HBASE-14123-v5.patch, 
> HBASE-14123-v6.patch, HBASE-14123-v7.patch, HBASE-14123-v9.patch
>
>
> Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15504) Fix Balancer in 1.3 not moving regions off overloaded regionserver

[Guanghao Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205597#comment-15205597
 ] 

Guanghao Zhang commented on HBASE-15504:


HBASE-14604 increase move cost when cluster regions number is bigger than 
maxMoves. Before HBASE-14604, it scale move cost between [0, 
cluster.numRegions] and it doesn't matter with maxMoves. But in our use case, 
we config small maxMoves because we don't want move too much region on our 
online serving cluster. HBASE-14604 will scale move cost between [0, 
Math.min(cluster.numRegions, maxMoves)]. Do you mind to share your config about 
maxMoves and cluster regions number?

> Fix Balancer in 1.3 not moving regions off overloaded regionserver
> --
>
> Key: HBASE-15504
> URL: https://issues.apache.org/jira/browse/HBASE-15504
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Elliott Clark
> Fix For: 1.3.0
>
>
> We pushed 1.3 to a couple of clusters. In some cases the regions were 
> assigned VERY un-evenly and the regions would not move after that.
> We ended up with one rs getting thousands of regions and most servers getting 
> 0. Running balancer would do nothing. The balancer would say that it couldn't 
> find a solution with less than the current cost.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15360) Fix flaky TestSimpleRpcScheduler

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205594#comment-15205594
 ] 

Duo Zhang commented on HBASE-15360:
---

Let's open a new issue for this? [~tedyu]
The patch uploaded here is a fix for testSoftAndHardQueueLimits.

> Fix flaky TestSimpleRpcScheduler
> 
>
> Key: HBASE-15360
> URL: https://issues.apache.org/jira/browse/HBASE-15360
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Mikhail Antonov
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15360.patch
>
>
> There were several flaky tests added there as part of HBASE-15306 and likely 
> HBASE-15136.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15411) Rewrite backup with Procedure V2

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205587#comment-15205587
 ] 

Enis Soztutar commented on HBASE-15411:
---

To those of us following the backup work, this patch comes on top of 
HBASE-14030 patch and moves the backup execution to the server side. Phase 2 
combined patch (HBASE-14123) comes on top of this. So ordering is HBASE-14030 
-> HBASE-15411 -> HBASE-14123. 

Phase 1 patch (HBASE-14030) is not committed yet, since I would like to see the 
actual fixes to move the logic to server side and make sure that the backup 
file layout and PB data structures, file manifests, etc, are as feature-proof 
as it can be. 

However, there is still some need to put this and HBASE-14123 on review board 
so that reviewers can get a sense of what is going on. Let me see what can be 
done there. 

> Rewrite backup with Procedure V2
> 
>
> Key: HBASE-15411
> URL: https://issues.apache.org/jira/browse/HBASE-15411
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15411-v1.txt, 15411-v11.txt, 15411-v12.txt, 
> 15411-v13.txt, 15411-v14.txt, 15411-v15.txt, 15411-v16.txt, 15411-v18.txt, 
> 15411-v3.txt, 15411-v5.txt, 15411-v6.txt, 15411-v7.txt, 15411-v9.txt, 
> FullTableBackupProcedure.java
>
>
> Currently full / incremental backup is driven by BackupHandler (see call() 
> method for flow).
> This issue is to rewrite the flow using Procedure V2.
> States (enum) for full / incremental backup would be introduced in 
> Backup.proto which correspond to the steps performed in BackupHandler#call().
> executeFromState() would pace the backup based on the current state.
> serializeStateData() / deserializeStateData() would be used to persist state 
> into procedure WAL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HBASE-15360) Fix flaky TestSimpleRpcScheduler

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu reopened HBASE-15360:


> Fix flaky TestSimpleRpcScheduler
> 
>
> Key: HBASE-15360
> URL: https://issues.apache.org/jira/browse/HBASE-15360
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Mikhail Antonov
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15360.patch
>
>
> There were several flaky tests added there as part of HBASE-15306 and likely 
> HBASE-15136.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15360) Fix flaky TestSimpleRpcScheduler

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205568#comment-15205568
 ] 

Ted Yu commented on HBASE-15360:


See also 
https://builds.apache.org/job/HBase-TRUNK_matrix/795/jdk=latest1.8,label=yahoo-not-h2/testReport/org.apache.hadoop.hbase.ipc/TestSimpleRpcScheduler/testCoDelScheduling/

> Fix flaky TestSimpleRpcScheduler
> 
>
> Key: HBASE-15360
> URL: https://issues.apache.org/jira/browse/HBASE-15360
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Mikhail Antonov
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15360.patch
>
>
> There were several flaky tests added there as part of HBASE-15306 and likely 
> HBASE-15136.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14030) HBase Backup/Restore Phase 1

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205520#comment-15205520
 ] 

Ted Yu commented on HBASE-14030:


BackupUtil.java is for backup related classes in hbase-server module.

BackupUtility.java is for client facing API.

If you have further suggestion on how they can be merged, that would be great.

> HBase Backup/Restore Phase 1
> 
>
> Key: HBASE-14030
> URL: https://issues.apache.org/jira/browse/HBASE-14030
> Project: HBase
>  Issue Type: Umbrella
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-14030-v0.patch, HBASE-14030-v1.patch, 
> HBASE-14030-v10.patch, HBASE-14030-v11.patch, HBASE-14030-v12.patch, 
> HBASE-14030-v13.patch, HBASE-14030-v14.patch, HBASE-14030-v15.patch, 
> HBASE-14030-v17.patch, HBASE-14030-v18.patch, HBASE-14030-v2.patch, 
> HBASE-14030-v20.patch, HBASE-14030-v21.patch, HBASE-14030-v22.patch, 
> HBASE-14030-v23.patch, HBASE-14030-v24.patch, HBASE-14030-v25.patch, 
> HBASE-14030-v26.patch, HBASE-14030-v27.patch, HBASE-14030-v28.patch, 
> HBASE-14030-v3.patch, HBASE-14030-v30.patch, HBASE-14030-v35.patch, 
> HBASE-14030-v37.patch, HBASE-14030-v4.patch, HBASE-14030-v5.patch, 
> HBASE-14030-v6.patch, HBASE-14030-v7.patch, HBASE-14030-v8.patch, 
> HBASE-14030.v38.patch, HBASE-14030.v39.patch, HBASE-14030.v40.patch, 
> HBASE-14030.v41.patch, HBASE-14030.v42.patch, hbase-14030_v36.patch
>
>
> This is the umbrella ticket for Backup/Restore Phase 1. See HBASE-7912 design 
> doc for the phase description.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14030) HBase Backup/Restore Phase 1

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205513#comment-15205513
 ] 

Enis Soztutar commented on HBASE-14030:
---

Thanks Ted for the short summaries in the patch iterations. It helps with 
keeping up with the code. 

bq. Patch v40 introduces BackupUtility (reviewed by Vlad offline) in the 
hbase-client module.
We now have {{BackupUtil}} and {{BackupUtility}}. There should be only one.  
./hbase-client/src/main/java/org/apache/hadoop/hbase/backup/BackupUtility.java
./hbase-server/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupUtil.java

> HBase Backup/Restore Phase 1
> 
>
> Key: HBASE-14030
> URL: https://issues.apache.org/jira/browse/HBASE-14030
> Project: HBase
>  Issue Type: Umbrella
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-14030-v0.patch, HBASE-14030-v1.patch, 
> HBASE-14030-v10.patch, HBASE-14030-v11.patch, HBASE-14030-v12.patch, 
> HBASE-14030-v13.patch, HBASE-14030-v14.patch, HBASE-14030-v15.patch, 
> HBASE-14030-v17.patch, HBASE-14030-v18.patch, HBASE-14030-v2.patch, 
> HBASE-14030-v20.patch, HBASE-14030-v21.patch, HBASE-14030-v22.patch, 
> HBASE-14030-v23.patch, HBASE-14030-v24.patch, HBASE-14030-v25.patch, 
> HBASE-14030-v26.patch, HBASE-14030-v27.patch, HBASE-14030-v28.patch, 
> HBASE-14030-v3.patch, HBASE-14030-v30.patch, HBASE-14030-v35.patch, 
> HBASE-14030-v37.patch, HBASE-14030-v4.patch, HBASE-14030-v5.patch, 
> HBASE-14030-v6.patch, HBASE-14030-v7.patch, HBASE-14030-v8.patch, 
> HBASE-14030.v38.patch, HBASE-14030.v39.patch, HBASE-14030.v40.patch, 
> HBASE-14030.v41.patch, HBASE-14030.v42.patch, hbase-14030_v36.patch
>
>
> This is the umbrella ticket for Backup/Restore Phase 1. See HBASE-7912 design 
> doc for the phase description.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15464) Flush / Compaction metrics revisited

[Enis Soztutar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HBASE-15464:
--
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed this to 1.3+. Thanks Elliott for the review. 

> Flush / Compaction metrics revisited
> 
>
> Key: HBASE-15464
> URL: https://issues.apache.org/jira/browse/HBASE-15464
> Project: HBase
>  Issue Type: Sub-task
>  Components: metrics
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: hbase-15464_v1.patch, hbase-15464_v2.patch, 
> hbase-15464_v3.patch, hbase-15464_v3.patch, hbase-15464_v4.patch
>
>
> We can add a couple of metrics related to flushes and compactions: 
>  - flush memstore and output file size histogram: This will allow seeing 
> whether we are flushing too early due to memory pressure, too many regions, 
> etc. Tracking flush memstore size vs output file size is useful in 
> understanding the block encoding compression benefits. 
>  - total flushed output bytes: This will allow to monitor the IO / throughput 
> from flushers. You can use this to set num flushers, flush throttle, etc. 
>  - smallCompactionQueueLength / large...: This is tracked, but not emitted 
> anymore due to a bug. 
>  - compaction time histogram: similar to flush time histogram, how long 
> compactions are taking. 
>  - compaction input num files / output num files histogram: How many files on 
> average we are compacting. Stripe compaction / date tiered compaction can use 
> the num output files metric. 
>  - compaction input / output data sizes histogram: How much data on average 
> we are compacting. 
>  - compaction input / output total bytes: Measure compaction IO / throughput. 
> measure write amplification, enables to set compaction throttle. 
>  - Breakdown for above for major compactions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15464) Flush / Compaction metrics revisited

[Enis Soztutar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HBASE-15464:
--
Attachment: hbase-15464_v4.patch

rebased v4 patch. 

I'll commit this shortly. I've ran the tests manually, and they pass. Other 
checks are fine from hadoopqa. 

> Flush / Compaction metrics revisited
> 
>
> Key: HBASE-15464
> URL: https://issues.apache.org/jira/browse/HBASE-15464
> Project: HBase
>  Issue Type: Sub-task
>  Components: metrics
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: hbase-15464_v1.patch, hbase-15464_v2.patch, 
> hbase-15464_v3.patch, hbase-15464_v3.patch, hbase-15464_v4.patch
>
>
> We can add a couple of metrics related to flushes and compactions: 
>  - flush memstore and output file size histogram: This will allow seeing 
> whether we are flushing too early due to memory pressure, too many regions, 
> etc. Tracking flush memstore size vs output file size is useful in 
> understanding the block encoding compression benefits. 
>  - total flushed output bytes: This will allow to monitor the IO / throughput 
> from flushers. You can use this to set num flushers, flush throttle, etc. 
>  - smallCompactionQueueLength / large...: This is tracked, but not emitted 
> anymore due to a bug. 
>  - compaction time histogram: similar to flush time histogram, how long 
> compactions are taking. 
>  - compaction input num files / output num files histogram: How many files on 
> average we are compacting. Stripe compaction / date tiered compaction can use 
> the num output files metric. 
>  - compaction input / output data sizes histogram: How much data on average 
> we are compacting. 
>  - compaction input / output total bytes: Measure compaction IO / throughput. 
> measure write amplification, enables to set compaction throttle. 
>  - Breakdown for above for major compactions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15513) hbase.hregion.memstore.chunkpool.maxsize is 0.0 by default

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205486#comment-15205486
 ] 

Vladimir Rodionov commented on HBASE-15513:
---

Server-side.

> hbase.hregion.memstore.chunkpool.maxsize is 0.0 by default
> --
>
> Key: HBASE-15513
> URL: https://issues.apache.org/jira/browse/HBASE-15513
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
>
> That results in excessive MemStoreLAB chunk allocations because we can not 
> reuse them. Not sure, why it has been disabled, by default. May be the code 
> has not been tested well?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15513) hbase.hregion.memstore.chunkpool.maxsize is 0.0 by default

[Vladimir Rodionov (JIRA)]

Vladimir Rodionov created HBASE-15513:
-

 Summary: hbase.hregion.memstore.chunkpool.maxsize is 0.0 by default
 Key: HBASE-15513
 URL: https://issues.apache.org/jira/browse/HBASE-15513
 Project: HBase
  Issue Type: Improvement
Affects Versions: 2.0.0
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
 Fix For: 2.0.0


That results in excessive MemStoreLAB chunk allocations because we can not 
reuse them. Not sure, why it has been disabled, by default. May be the code has 
not been tested well?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15492) Memory usage optimizations (Write path)

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-15492:
--
Summary: Memory usage optimizations (Write path)  (was: Memory usage 
optimizations)

> Memory usage optimizations (Write path)
> ---
>
> Key: HBASE-15492
> URL: https://issues.apache.org/jira/browse/HBASE-15492
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: baseline.jfr, pre-HBASE-15479.jfr
>
>
> This is master ticket for all memory optimization tasks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15512) Avoid cell allocation on Store.add(Cell)

[Vladimir Rodionov (JIRA)]

Vladimir Rodionov created HBASE-15512:
-

 Summary: Avoid cell allocation on Store.add(Cell)
 Key: HBASE-15512
 URL: https://issues.apache.org/jira/browse/HBASE-15512
 Project: HBase
  Issue Type: Improvement
  Components: Performance
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov


AbstractMemStore.add(Cell) clones the cell. No reason for that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15502) Skeleton unit test to copy/paste

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205456#comment-15205456
 ] 

Hudson commented on HBASE-15502:


FAILURE: Integrated in HBase-Trunk_matrix #795 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/795/])
HBASE-15502 Skeleton unit test to copy/paste (stack: rev 
1123be9181155c700f28ad04050a5b63b06d7e63)
* src/main/asciidoc/_chapters/developer.adoc


> Skeleton unit test to copy/paste
> 
>
> Key: HBASE-15502
> URL: https://issues.apache.org/jira/browse/HBASE-15502
> Project: HBase
>  Issue Type: Task
>  Components: documentation
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: 15502.patch
>
>
> Add to our docs a skeleton unit test to copy/paste with all the vitals 
> already filled out such as category and categorybasetimeout @Rule.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15460) Fix infer issues in hbase-common

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205457#comment-15205457
 ] 

Hudson commented on HBASE-15460:


FAILURE: Integrated in HBase-Trunk_matrix #795 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/795/])
HBASE-15460 Fix infer issues in hbase-common (eclark: rev 
75252af3a983966de9a488389987a773b59ae4da)
* 
hbase-common/src/main/java/org/apache/hadoop/hbase/util/test/LoadTestKVGenerator.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/util/JVM.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java
* 
hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java
* 
hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultDecodingContext.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/nio/SingleByteBuff.java
* 
hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/util/Base64.java


> Fix infer issues in hbase-common
> 
>
> Key: HBASE-15460
> URL: https://issues.apache.org/jira/browse/HBASE-15460
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15460-v1.patch, HBASE-15460.patch, 
> hbase-common.infer
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205441#comment-15205441
 ] 

Hudson commented on HBASE-15433:


FAILURE: Integrated in HBase-1.1-JDK8 #1772 (See 
[https://builds.apache.org/job/HBase-1.1-JDK8/1772/])
HBASE-15433 SnapshotManager#restoreSnapshot not update table and region (tedyu: 
rev 40e19e80b2363c899f9b8cba98f3c2326d6329d1)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceStateManager.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceAuditor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java


> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15433-branch-1-v1.patch, 
> HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15511) ClusterStatus should be able to return responses by scope

[Esteban Gutierrez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Esteban Gutierrez updated HBASE-15511:
--
Description: 
The current ClusterStatus response returns too much information about the load 
per region and replication cluster wide. Sometimes that response can be quite 
large (10s or 100s of MBs) and methods like getServerSize() or 
getRegionsCount() don't really need the full response. One possibility is to 
provide a scope (or filter) for the ClusterStatus requests to limit the 
response back to the client.


  was:The current ClusterStatus returns too much information about the load per 
regions and replication cluster wide. Sometimes that response can be quite 
large (10s or 100s of MBs) and methods like getServerSize() or 
getRegionsCount() 


> ClusterStatus should be able to return responses by scope
> -
>
> Key: HBASE-15511
> URL: https://issues.apache.org/jira/browse/HBASE-15511
> Project: HBase
>  Issue Type: Improvement
>Reporter: Esteban Gutierrez
>
> The current ClusterStatus response returns too much information about the 
> load per region and replication cluster wide. Sometimes that response can be 
> quite large (10s or 100s of MBs) and methods like getServerSize() or 
> getRegionsCount() don't really need the full response. One possibility is to 
> provide a scope (or filter) for the ClusterStatus requests to limit the 
> response back to the client.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15360) Fix flaky TestSimpleRpcScheduler

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205426#comment-15205426
 ] 

Ted Yu commented on HBASE-15360:


>From 
>https://builds.apache.org/job/HBase-1.4/40/jdk=latest1.7,label=yahoo-not-h2/consoleFull
> :
{code}
Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 13.031 sec <<< 
FAILURE! - in org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler
testCoDelScheduling(org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler)  Time 
elapsed: 1.225 sec  <<< FAILURE!
java.lang.AssertionError: None of these calls should have been discarded 
expected:<0> but was:<2>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at 
org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testCoDelScheduling(TestSimpleRpcScheduler.java:384)

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 14.167 sec <<< 
FAILURE! - in org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler
testCoDelScheduling(org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler)  Time 
elapsed: 1.063 sec  <<< FAILURE!
java.lang.AssertionError: None of these calls should have been discarded 
expected:<0> but was:<1>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at 
org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testCoDelScheduling(TestSimpleRpcScheduler.java:384)
{code}

> Fix flaky TestSimpleRpcScheduler
> 
>
> Key: HBASE-15360
> URL: https://issues.apache.org/jira/browse/HBASE-15360
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Mikhail Antonov
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15360.patch
>
>
> There were several flaky tests added there as part of HBASE-15306 and likely 
> HBASE-15136.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15511) ClusterStatus should be able to return responses by scope

[Esteban Gutierrez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Esteban Gutierrez updated HBASE-15511:
--
Description: The current ClusterStatus returns too much information about 
the load per regions and replication cluster wide. Sometimes that response can 
be quite large (10s or 100s of MBs) and methods like 

> ClusterStatus should be able to return responses by scope
> -
>
> Key: HBASE-15511
> URL: https://issues.apache.org/jira/browse/HBASE-15511
> Project: HBase
>  Issue Type: Improvement
>Reporter: Esteban Gutierrez
>
> The current ClusterStatus returns too much information about the load per 
> regions and replication cluster wide. Sometimes that response can be quite 
> large (10s or 100s of MBs) and methods like 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15511) ClusterStatus should be able to return responses by scope

[Esteban Gutierrez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Esteban Gutierrez updated HBASE-15511:
--
Description: The current ClusterStatus returns too much information about 
the load per regions and replication cluster wide. Sometimes that response can 
be quite large (10s or 100s of MBs) and methods like getServerSize() or 
getRegionsCount()   (was: The current ClusterStatus returns too much 
information about the load per regions and replication cluster wide. Sometimes 
that response can be quite large (10s or 100s of MBs) and methods like )

> ClusterStatus should be able to return responses by scope
> -
>
> Key: HBASE-15511
> URL: https://issues.apache.org/jira/browse/HBASE-15511
> Project: HBase
>  Issue Type: Improvement
>Reporter: Esteban Gutierrez
>
> The current ClusterStatus returns too much information about the load per 
> regions and replication cluster wide. Sometimes that response can be quite 
> large (10s or 100s of MBs) and methods like getServerSize() or 
> getRegionsCount() 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14147) REST Support for Namespaces

[Misty Stanley-Jones (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205420#comment-15205420
 ] 

Misty Stanley-Jones commented on HBASE-14147:
-

Hey [~mwarhaftig] and [~apurtell], I cannot get the POST endpoint to work and 
create a namespace. I'm using this syntax:
{code}
curl -vi -X POST -H "Accept: text/xml" -H "Content-Type: text/xml"  
"http://my_server:20550/namespaces/new_ns2/;
{code}
I have tried with and without the trailing slash. The response I get is:

{code}
< HTTP/1.1 400 Bad Request
HTTP/1.1 400 Bad Request
< Content-Type: text/html; charset=iso-8859-1
Content-Type: text/html; charset=iso-8859-1
< Cache-Control: must-revalidate,no-cache,no-store
Cache-Control: must-revalidate,no-cache,no-store
< Content-Length: 1387
Content-Length: 1387
{code}

I'm going to try this on Vanilla HBase tomorrow, as this is on CDH that I'm 
seeing the error. I'll let you know, but in the meantime, if you could spot 
what I'm doing wrong that would be helpful. I don't seem to be getting any 
helpful logs anywhere either. [~stack] was helping me out and we didn't get 
anywhere.

> REST Support for Namespaces
> ---
>
> Key: HBASE-14147
> URL: https://issues.apache.org/jira/browse/HBASE-14147
> Project: HBase
>  Issue Type: Sub-task
>  Components: REST
>Affects Versions: 1.1.1
>Reporter: Rick Kellogg
>Assignee: Matt Warhaftig
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.15
>
> Attachments: SuccessfulLocalBuild.txt, hbase-14147-v1.patch, 
> hbase-14147-v1.patch, hbase-14147-v1.patch
>
>
> Expand REST services to include addition features:
> * Create namespace
> * Alter namespace
> * Describe namespace
> * Drop namespace
> * List tables in a specific namespace
> * List all namespaces.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15460) Fix infer issues in hbase-common

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205415#comment-15205415
 ] 

Hudson commented on HBASE-15460:


FAILURE: Integrated in HBase-1.4 #40 (See 
[https://builds.apache.org/job/HBase-1.4/40/])
HBASE-15460 Fix infer issues in hbase-common (eclark: rev 
295e5cbcf038bbd86cd5dbc05ae652f09b5c5bdf)
* hbase-common/src/main/java/org/apache/hadoop/hbase/util/Base64.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/util/JVM.java
* 
hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java
* 
hbase-common/src/main/java/org/apache/hadoop/hbase/util/test/LoadTestKVGenerator.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java
* 
hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java


> Fix infer issues in hbase-common
> 
>
> Key: HBASE-15460
> URL: https://issues.apache.org/jira/browse/HBASE-15460
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15460-v1.patch, HBASE-15460.patch, 
> hbase-common.infer
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15511) ClusterStatus should be able to return responses by scope

[Esteban Gutierrez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Esteban Gutierrez updated HBASE-15511:
--
Summary: ClusterStatus should be able to return responses by scope  (was: 
ClusterStatus should be able )

> ClusterStatus should be able to return responses by scope
> -
>
> Key: HBASE-15511
> URL: https://issues.apache.org/jira/browse/HBASE-15511
> Project: HBase
>  Issue Type: Improvement
>Reporter: Esteban Gutierrez
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)