[jira] [Commented] (HBASE-17167) Pass mvcc to client when scan
[
https://issues.apache.org/jira/browse/HBASE-17167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704545#comment-15704545
]
stack commented on HBASE-17167:
---
Yeah, the protoc error says...
{code}
...
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 18.296s
[INFO] Finished at: Tue Nov 29 04:20:52 UTC 2016
[INFO] Final Memory: 69M/1600M
[INFO]
[WARNING] The requested profile "compile-protobuf" could not be activated
because it does not exist.
{code}
... complaining about missing protocs. We need to fix. HBASE-17119. There is
another issue for skipping findbugs.
I looked at the patch +1 (thanks for the gymnastics that tried to keep setting
mvcc package private). TestMvccConsistentScanner could go into the atomicity
tests. Can do that later.
+1
> Pass mvcc to client when scan
> -
>
> Key: HBASE-17167
> URL: https://issues.apache.org/jira/browse/HBASE-17167
> Project: HBase
> Issue Type: Sub-task
> Components: Client, scan
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17167-v1.patch, HBASE-17167-v2.patch,
> HBASE-17167-v3.patch, HBASE-17167-v4.patch, HBASE-17167.patch
>
>
> For the current implementation, if we use batch or allowPartial when scan,
> then the row level atomic can not be guaranteed if we need to restart a scan
> in the middle of a record due to region move or something else.
> We can return the mvcc used to open scanner to client and client could use
> this mvcc to restart a scan to get row level atomic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17048) Calcuate suitable ByteBuf size when allocating send buffer in FanOutOneBlockAsyncDFSOutput
[ https://issues.apache.org/jira/browse/HBASE-17048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704513#comment-15704513 ] Hudson commented on HBASE-17048: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2039 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2039/]) HBASE-17048 Calcuate suitable ByteBuf size when allocating send buffer (ramkrishna: rev 51d9bac42b30c2b1a81094970a1ce50f70de3192) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutput.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/asyncfs/TestFanOutOneBlockAsyncDFSOutput.java > Calcuate suitable ByteBuf size when allocating send buffer in > FanOutOneBlockAsyncDFSOutput > -- > > Key: HBASE-17048 > URL: https://issues.apache.org/jira/browse/HBASE-17048 > Project: HBase > Issue Type: Sub-task > Components: wal >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-17012_3.patch, HBASE-17048.patch, > HBASE-17048_1.patch, HBASE-17048_3.patch > > > As [~ram_krish] mentioned in HBASE-17021 > https://issues.apache.org/jira/browse/HBASE-17021?focusedCommentId=15646938=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15646938 > The default ByteBuf size is 256B which is too small. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16561) Add metrics about read/write/scan queue length and active read/write/scan handler count
[ https://issues.apache.org/jira/browse/HBASE-16561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704512#comment-15704512 ] Hudson commented on HBASE-16561: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2039 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2039/]) HBASE-16561 Add metrics about read/write/scan queue length and active (zhangduo: rev cc03f7ad5320d9b91cd65e0630501d08d341ad74) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/DelegatingRpcScheduler.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RWQueueRpcExecutor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/FastPathBalancedQueueRpcExecutor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestRpcMetrics.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcScheduler.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/FifoRpcScheduler.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerWrapperImpl.java * (edit) hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSourceImpl.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSource.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerWrapper.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerWrapperStub.java > Add metrics about read/write/scan queue length and active read/write/scan > handler count > --- > > Key: HBASE-16561 > URL: https://issues.apache.org/jira/browse/HBASE-16561 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, metrics >Affects Versions: 2.0.0, 1.4.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16561-branch-1.patch, HBASE-16561-v1.patch, > HBASE-16561.patch > > > Now there are only metrics about total queue length and active rpc handler > count. But in the RWQueueRpcExecutor, there are different queues and handlers > for read/write/scan request. I thought it is necessary to add more metrics > for RWQueueRpcExecutor. When use it in production cluster, we can adjust the > config of queues and handlers according to the metrics. > Review url: https://reviews.apache.org/r/54072/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-17086) Add comments to explain why Cell#getTagsLength() returns an int, rather than a short
[ https://issues.apache.org/jira/browse/HBASE-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John resolved HBASE-17086. Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Thanks for the patch Xiang Li > Add comments to explain why Cell#getTagsLength() returns an int, rather than > a short > > > Key: HBASE-17086 > URL: https://issues.apache.org/jira/browse/HBASE-17086 > Project: HBase > Issue Type: Improvement > Components: Interface >Affects Versions: 2.0.0 >Reporter: Xiang Li >Assignee: Xiang Li >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-17086.master.000.patch, > HBASE-17086.master.001.patch > > > In the Cell interface, getTagsLength() returns a int > But in the KeyValue implementation, tags length is of 2 bytes. Also in > ExtendedCell, when explaining the KeyValue format, tags length is stated to > be 2 bytes > Any plan to update Cell interface to make getTagsLength() returns a short ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16912) TEST: update HBaseTestingUtility to avoid direct use of filesystem / legacy implementation
[ https://issues.apache.org/jira/browse/HBASE-16912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-16912: Resolution: Fixed Status: Resolved (was: Patch Available) > TEST: update HBaseTestingUtility to avoid direct use of filesystem / legacy > implementation > -- > > Key: HBASE-16912 > URL: https://issues.apache.org/jira/browse/HBASE-16912 > Project: HBase > Issue Type: Sub-task > Components: Filesystem Integration >Affects Versions: hbase-14439 >Reporter: Sean Busbey >Assignee: Appy > Fix For: hbase-14439 > > Attachments: HBASE-16912.hbase-14439.001.patch, > HBASE-16912.hbase-14439.002.patch, HBASE-16912.hbase-14439.002.review, > HBASE-16912.hbase-14439.003.patch, fs-redo-HBTU.patch > > > Update the HBaseTestingUtility to ensure it relies on MasterStorage / > RegionStorage APIs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17189) TestMasterObserver#wasModifyTableActionCalled uses wrong variables
[
https://issues.apache.org/jira/browse/HBASE-17189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Yuan Jiang updated HBASE-17189:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 1.4.0
2.0.0
Status: Resolved (was: Patch Available)
> TestMasterObserver#wasModifyTableActionCalled uses wrong variables
> --
>
> Key: HBASE-17189
> URL: https://issues.apache.org/jira/browse/HBASE-17189
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Stephen Yuan Jiang
>Assignee: Stephen Yuan Jiang
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17189.v1-master.patch
>
>
> TestMasterObserver#wasModifyTableActionCalled() and
> TestMasterObserver#wasModifyTableActionCalledOnly() uses
> {{preModifyColumnFamilyActionCalled}} and
> {{postCompletedModifyColumnFamilyActionCalled}} members, which are wrong.
> Instead it should use {{preModifyTableActionCalled}} and
> {{postCompletedModifyTableActionCalled}}. This probably was caused by
> copy-and-paste mistake.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15787) Change the flush related heuristics to work with offheap size configured
[
https://issues.apache.org/jira/browse/HBASE-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704484#comment-15704484
]
ramkrishna.s.vasudevan commented on HBASE-15787:
ABOVE_OFFHEAP_HIGHER_MARK - I need this to decide what caused the flush. I
don't allow the heap tuner to tune the offheap but only uses this metric to
know if I am allowed to reduce the memstore heap size.
I will try to remove this as part of my TODO
{code}
// See if we really want this?? Will change or leave it
+// after discussion
{code}
bq.// TODO : add support for offheap metrics
Metric I will add a new JIRA to cover offheap things. So that is why there is a
TODO. Will post in RB after adding some comments.
> Change the flush related heuristics to work with offheap size configured
>
>
> Key: HBASE-15787
> URL: https://issues.apache.org/jira/browse/HBASE-15787
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-15787.patch, HBASE-15787_1.patch
>
>
> With offheap MSLAB in place we may have to change the flush related
> heuristics to work with offheap size configured rather than the java heap
> size.
> Since we now have clear seperation of the memstore data size and memstore
> heap size, for offheap memstore
> -> Decide if the global.offheap.memstore.size is breached for blocking
> updates and force flushes.
> -> If the onheap global.memstore.size is breached (due to heap overhead) even
> then block updates and force flushes.
> -> The global.memstore.size.lower.limit is now by default 95% of the
> global.memstore.size. So now we apply this 95% on the
> global.offheap.memstore.size and also on global.memstore.size (as it was done
> for onheap case).
> -> We will have new FlushTypes introduced
> {code}
> ABOVE_ONHEAP_LOWER_MARK, /* happens due to lower mark breach of onheap
> memstore settings
> An offheap memstore can even breach the
> onheap_lower_mark*/
> ABOVE_ONHEAP_HIGHER_MARK,/* happens due to higher mark breach of onheap
> memstore settings
> An offheap memstore can even breach the
> onheap_higher_mark*/
> ABOVE_OFFHEAP_LOWER_MARK,/* happens due to lower mark breach of offheap
> memstore settings*/
> ABOVE_OFFHEAP_HIGHER_MARK;
> {code}
> -> regionServerAccounting does all the accounting.
> -> HeapMemoryTuner is what is litte tricky here. First thing to note is that
> at no point it will try to increase or decrease the
> global.offheap.memstore.size. If there is a heap pressure then it will try to
> increase the memstore heap limit.
> In case of offheap memstore there is always a chance that the heap pressure
> does not increase. In that case we could ideally decrease the heap limit for
> memstore. The current logic of heapmemory tuner is such that things will
> naturally settle down. But on discussion what we thought is let us include
> the flush count that happens due to offheap pressure but give that a lesser
> weightage and thus ensure that the initial decrease on memstore heap limit
> does not happen. Currently that fraction is set as 0.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15787) Change the flush related heuristics to work with offheap size configured
[
https://issues.apache.org/jira/browse/HBASE-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704459#comment-15704459
]
stack commented on HBASE-15787:
---
Looking at the patch, can DefaultHeapMemoryTuner be made standalone so it can
be tested in isolation? I'd think getting initial values right would be easy
enough but would be cool if we could run a scenario and watch how the feedback
loop develops over a period of time... see if it settles or if it does a Tacoma
Narrows Bridge behavior.
RegionServerAccounting is memory only? Should it say that it is memory only?
RegionServerMemoryAccounting? Or just MemoryAccounting since the RegionServer
is a given?
This seems pretty important given as we don't have a good way at looking at
offheap:
// TODO : add support for offheap metrics
Why am I doing ABOVE_OFFHEAP_HIGHER_MARK in the HeapMemoryManager (unless
HeapMemoryManager is for both onheap and offheap?)
I gave it a pass.
> Change the flush related heuristics to work with offheap size configured
>
>
> Key: HBASE-15787
> URL: https://issues.apache.org/jira/browse/HBASE-15787
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-15787.patch, HBASE-15787_1.patch
>
>
> With offheap MSLAB in place we may have to change the flush related
> heuristics to work with offheap size configured rather than the java heap
> size.
> Since we now have clear seperation of the memstore data size and memstore
> heap size, for offheap memstore
> -> Decide if the global.offheap.memstore.size is breached for blocking
> updates and force flushes.
> -> If the onheap global.memstore.size is breached (due to heap overhead) even
> then block updates and force flushes.
> -> The global.memstore.size.lower.limit is now by default 95% of the
> global.memstore.size. So now we apply this 95% on the
> global.offheap.memstore.size and also on global.memstore.size (as it was done
> for onheap case).
> -> We will have new FlushTypes introduced
> {code}
> ABOVE_ONHEAP_LOWER_MARK, /* happens due to lower mark breach of onheap
> memstore settings
> An offheap memstore can even breach the
> onheap_lower_mark*/
> ABOVE_ONHEAP_HIGHER_MARK,/* happens due to higher mark breach of onheap
> memstore settings
> An offheap memstore can even breach the
> onheap_higher_mark*/
> ABOVE_OFFHEAP_LOWER_MARK,/* happens due to lower mark breach of offheap
> memstore settings*/
> ABOVE_OFFHEAP_HIGHER_MARK;
> {code}
> -> regionServerAccounting does all the accounting.
> -> HeapMemoryTuner is what is litte tricky here. First thing to note is that
> at no point it will try to increase or decrease the
> global.offheap.memstore.size. If there is a heap pressure then it will try to
> increase the memstore heap limit.
> In case of offheap memstore there is always a chance that the heap pressure
> does not increase. In that case we could ideally decrease the heap limit for
> memstore. The current logic of heapmemory tuner is such that things will
> naturally settle down. But on discussion what we thought is let us include
> the flush count that happens due to offheap pressure but give that a lesser
> weightage and thus ensure that the initial decrease on memstore heap limit
> does not happen. Currently that fraction is set as 0.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17187) DoNotRetryExceptions from coprocessors should bubble up to the application
[
https://issues.apache.org/jira/browse/HBASE-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704457#comment-15704457
]
Anoop Sam John commented on HBASE-17187:
UnknownScannerException etc are DNRIOE only. So addition of this
{code}
if (e instanceof DoNotRetryIOException) {
3013throw e;
3014 }
{code}
means we will not throw back ScannerResetException now?
So which all IO exception will come under and converted to
ScannerResetException now? Seems we wanted to consider
UnknownScannerException only?
{code}
if (VersionInfoUtil.hasMinimumVersion(context.getClientVersionInfo(), 1, 4)) {
// 1.4.0+ clients know how to handle
throw new ScannerResetException("Scanner is closed on the
server-side", e);
} else {
// older clients do not know about SRE. Just throw USE, which they
will handle
throw new UnknownScannerException("Throwing UnknownScannerException
to reset the client"
+ " scanner state for clients older than 1.3.", e);
}
{code}
Also this is a resent fix
{code}
if (e instanceof CorruptHFileException || e instanceof FileNotFoundException) {
throw new DoNotRetryIOException(e);
}
{code}
Now after ur code addition CorruptHFileException is handled by not FNFE.
Would have been best if we converted FNFE to HBase specific (HFileNotFoundE)
and thrown up the layers. But not directly related to this patch any way.
> DoNotRetryExceptions from coprocessors should bubble up to the application
> --
>
> Key: HBASE-17187
> URL: https://issues.apache.org/jira/browse/HBASE-17187
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-17187_v1.patch
>
>
> In HBASE-16604, we fixed a case where scanner retries was causing the scan to
> miss some data in case the scanner is left with a dirty state (like a
> half-seeked KVHeap).
> The patch introduced a minor compatibility issue, because now if a
> coprocessor throws DNRIOE, we still retry the ClientScanner indefinitely.
> The test {{ServerExceptionIT}} in Phoenix is failing because of this with
> HBASE-16604.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16261) MultiHFileOutputFormat Enhancement
[
https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704455#comment-15704455
]
Hadoop QA commented on HBASE-16261:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 2s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 17s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 110m 0s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 158m 9s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12840800/HBASE-16261-V4.patch |
| JIRA Issue | HBASE-16261 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 36fa59b31f8a 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 51d9bac |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4668/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4668/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> MultiHFileOutputFormat Enhancement
>
>
> Key: HBASE-16261
> URL: https://issues.apache.org/jira/browse/HBASE-16261
> Project: HBase
> Issue Type: Sub-task
> Components: hbase
>Affects Versions: 2.0.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16261-V1.patch,
[jira] [Created] (HBASE-17191) Make use of UnsafeByteOperations#unsafeWrap(ByteBuffer buffer) in PBUtil#toCell(Cell cell)
ramkrishna.s.vasudevan created HBASE-17191: -- Summary: Make use of UnsafeByteOperations#unsafeWrap(ByteBuffer buffer) in PBUtil#toCell(Cell cell) Key: HBASE-17191 URL: https://issues.apache.org/jira/browse/HBASE-17191 Project: HBase Issue Type: Improvement Affects Versions: 2.0.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Since we now have support for BBs in UnsafeByteOperations#unsafeWrap(ByteBuffer) . So for the non - java clients while creating the PB result we could avoid the copy to an temp array. Since we have a support to write to BB in ByteOutput having a result backed by Bytebuffer should be fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17186) MasterProcedureTestingUtility#testRecoveryAndDoubleExecution displays stale procedure state info
[ https://issues.apache.org/jira/browse/HBASE-17186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-17186: --- Hadoop Flags: Reviewed > MasterProcedureTestingUtility#testRecoveryAndDoubleExecution displays stale > procedure state info > > > Key: HBASE-17186 > URL: https://issues.apache.org/jira/browse/HBASE-17186 > Project: HBase > Issue Type: Bug > Components: proc-v2, test >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-17186.v1-master.patch, HBASE-17186.v2-master.patch > > > MasterProcedureTestingUtility#testRecoveryAndDoubleExecution get the > procedure information at the beginning of the function, but never updates the > information. As procedure executes and moves to new state, it still log the > stale state information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17186) MasterProcedureTestingUtility#testRecoveryAndDoubleExecution displays stale procedure state info
[ https://issues.apache.org/jira/browse/HBASE-17186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-17186: --- Affects Version/s: 2.0.0 > MasterProcedureTestingUtility#testRecoveryAndDoubleExecution displays stale > procedure state info > > > Key: HBASE-17186 > URL: https://issues.apache.org/jira/browse/HBASE-17186 > Project: HBase > Issue Type: Bug > Components: proc-v2, test >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-17186.v1-master.patch, HBASE-17186.v2-master.patch > > > MasterProcedureTestingUtility#testRecoveryAndDoubleExecution get the > procedure information at the beginning of the function, but never updates the > information. As procedure executes and moves to new state, it still log the > stale state information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17186) MasterProcedureTestingUtility#testRecoveryAndDoubleExecution displays stale procedure state info
[ https://issues.apache.org/jira/browse/HBASE-17186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-17186: --- Fix Version/s: 2.0.0 > MasterProcedureTestingUtility#testRecoveryAndDoubleExecution displays stale > procedure state info > > > Key: HBASE-17186 > URL: https://issues.apache.org/jira/browse/HBASE-17186 > Project: HBase > Issue Type: Bug > Components: proc-v2, test >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-17186.v1-master.patch, HBASE-17186.v2-master.patch > > > MasterProcedureTestingUtility#testRecoveryAndDoubleExecution get the > procedure information at the beginning of the function, but never updates the > information. As procedure executes and moves to new state, it still log the > stale state information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17186) MasterProcedureTestingUtility#testRecoveryAndDoubleExecution displays stale procedure state info
[ https://issues.apache.org/jira/browse/HBASE-17186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-17186: --- Resolution: Fixed Status: Resolved (was: Patch Available) > MasterProcedureTestingUtility#testRecoveryAndDoubleExecution displays stale > procedure state info > > > Key: HBASE-17186 > URL: https://issues.apache.org/jira/browse/HBASE-17186 > Project: HBase > Issue Type: Bug > Components: proc-v2, test >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-17186.v1-master.patch, HBASE-17186.v2-master.patch > > > MasterProcedureTestingUtility#testRecoveryAndDoubleExecution get the > procedure information at the beginning of the function, but never updates the > information. As procedure executes and moves to new state, it still log the > stale state information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17086) Add comments to explain why Cell#getTagsLength() returns an int, rather than a short
[ https://issues.apache.org/jira/browse/HBASE-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiang Li updated HBASE-17086: - Attachment: HBASE-17086.master.001.patch > Add comments to explain why Cell#getTagsLength() returns an int, rather than > a short > > > Key: HBASE-17086 > URL: https://issues.apache.org/jira/browse/HBASE-17086 > Project: HBase > Issue Type: Improvement > Components: Interface >Affects Versions: 2.0.0 >Reporter: Xiang Li >Assignee: Xiang Li >Priority: Minor > Attachments: HBASE-17086.master.000.patch, > HBASE-17086.master.001.patch > > > In the Cell interface, getTagsLength() returns a int > But in the KeyValue implementation, tags length is of 2 bytes. Also in > ExtendedCell, when explaining the KeyValue format, tags length is stated to > be 2 bytes > Any plan to update Cell interface to make getTagsLength() returns a short ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17086) Add comments to explain why Cell#getTagsLength() returns an int, rather than a short
[ https://issues.apache.org/jira/browse/HBASE-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704445#comment-15704445 ] Xiang Li commented on HBASE-17086: -- Hi [~anoop.hbase], I see, thanks! Patch 001 is uploaded to address your comments. > Add comments to explain why Cell#getTagsLength() returns an int, rather than > a short > > > Key: HBASE-17086 > URL: https://issues.apache.org/jira/browse/HBASE-17086 > Project: HBase > Issue Type: Improvement > Components: Interface >Affects Versions: 2.0.0 >Reporter: Xiang Li >Assignee: Xiang Li >Priority: Minor > Attachments: HBASE-17086.master.000.patch, > HBASE-17086.master.001.patch > > > In the Cell interface, getTagsLength() returns a int > But in the KeyValue implementation, tags length is of 2 bytes. Also in > ExtendedCell, when explaining the KeyValue format, tags length is stated to > be 2 bytes > Any plan to update Cell interface to make getTagsLength() returns a short ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16904) remove directory layout / fs references from snapshots
[
https://issues.apache.org/jira/browse/HBASE-16904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-16904:
Attachment: HBASE-16904-hbase-14439.v3.patch
-03
- adding addendum for the asf license and javadocs. commented things to get
test-compile to pass.
> remove directory layout / fs references from snapshots
> --
>
> Key: HBASE-16904
> URL: https://issues.apache.org/jira/browse/HBASE-16904
> Project: HBase
> Issue Type: Sub-task
> Components: Filesystem Integration
>Reporter: Sean Busbey
>Assignee: Umesh Agashe
> Attachments: HBASE-16904-hbase-14439.v1.patch,
> HBASE-16904-hbase-14439.v2.patch, HBASE-16904-hbase-14439.v3.patch
>
>
> ensure snapshot code works through the MasterStorage / RegionStorage APIs and
> not directly on backing filesystem.
> {code}
> hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java
> hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java
> hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotDescriptionUtils.java
> hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotInfo.java
> hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotManifest.java
> hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotManifestV1.java
> hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotManifestV2.java
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotFileCache.java
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotHFileCleaner.java
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15787) Change the flush related heuristics to work with offheap size configured
[
https://issues.apache.org/jira/browse/HBASE-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704421#comment-15704421
]
ramkrishna.s.vasudevan commented on HBASE-15787:
I will put this up on RB. Actually the heap memory tuner is smart enough such
that it is not doing reduction 1% at a time instead it does in steps. The step
starts from 0.00125f upto 0.04f. So this ensures that any reduction or increase
is not drastic.
So in this patch if there is offheap pressure flushes alone and there is a need
to increase the block cache size - we try to reduce the heap memstore size
using the minimum step size (0.00125).
And also the heap tuner is also smart enough to decide if it needs to be
neutral if the inputs that it receives does not allow it to decide what to do
and eventually things settle down. Thank you.
> Change the flush related heuristics to work with offheap size configured
>
>
> Key: HBASE-15787
> URL: https://issues.apache.org/jira/browse/HBASE-15787
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-15787.patch, HBASE-15787_1.patch
>
>
> With offheap MSLAB in place we may have to change the flush related
> heuristics to work with offheap size configured rather than the java heap
> size.
> Since we now have clear seperation of the memstore data size and memstore
> heap size, for offheap memstore
> -> Decide if the global.offheap.memstore.size is breached for blocking
> updates and force flushes.
> -> If the onheap global.memstore.size is breached (due to heap overhead) even
> then block updates and force flushes.
> -> The global.memstore.size.lower.limit is now by default 95% of the
> global.memstore.size. So now we apply this 95% on the
> global.offheap.memstore.size and also on global.memstore.size (as it was done
> for onheap case).
> -> We will have new FlushTypes introduced
> {code}
> ABOVE_ONHEAP_LOWER_MARK, /* happens due to lower mark breach of onheap
> memstore settings
> An offheap memstore can even breach the
> onheap_lower_mark*/
> ABOVE_ONHEAP_HIGHER_MARK,/* happens due to higher mark breach of onheap
> memstore settings
> An offheap memstore can even breach the
> onheap_higher_mark*/
> ABOVE_OFFHEAP_LOWER_MARK,/* happens due to lower mark breach of offheap
> memstore settings*/
> ABOVE_OFFHEAP_HIGHER_MARK;
> {code}
> -> regionServerAccounting does all the accounting.
> -> HeapMemoryTuner is what is litte tricky here. First thing to note is that
> at no point it will try to increase or decrease the
> global.offheap.memstore.size. If there is a heap pressure then it will try to
> increase the memstore heap limit.
> In case of offheap memstore there is always a chance that the heap pressure
> does not increase. In that case we could ideally decrease the heap limit for
> memstore. The current logic of heapmemory tuner is such that things will
> naturally settle down. But on discussion what we thought is let us include
> the flush count that happens due to offheap pressure but give that a lesser
> weightage and thus ensure that the initial decrease on memstore heap limit
> does not happen. Currently that fraction is set as 0.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15787) Change the flush related heuristics to work with offheap size configured
[
https://issues.apache.org/jira/browse/HBASE-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704403#comment-15704403
]
stack commented on HBASE-15787:
---
One note is that we should allow specifying less than full percentages and/or
finder granularity decimals. 1% of 100G is 1G which is a lot of RAM. Can we add
being able to specify to two decimal places in percntages and four if decimal
at least doing heap calculations?
Is this up on rb?
> Change the flush related heuristics to work with offheap size configured
>
>
> Key: HBASE-15787
> URL: https://issues.apache.org/jira/browse/HBASE-15787
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-15787.patch, HBASE-15787_1.patch
>
>
> With offheap MSLAB in place we may have to change the flush related
> heuristics to work with offheap size configured rather than the java heap
> size.
> Since we now have clear seperation of the memstore data size and memstore
> heap size, for offheap memstore
> -> Decide if the global.offheap.memstore.size is breached for blocking
> updates and force flushes.
> -> If the onheap global.memstore.size is breached (due to heap overhead) even
> then block updates and force flushes.
> -> The global.memstore.size.lower.limit is now by default 95% of the
> global.memstore.size. So now we apply this 95% on the
> global.offheap.memstore.size and also on global.memstore.size (as it was done
> for onheap case).
> -> We will have new FlushTypes introduced
> {code}
> ABOVE_ONHEAP_LOWER_MARK, /* happens due to lower mark breach of onheap
> memstore settings
> An offheap memstore can even breach the
> onheap_lower_mark*/
> ABOVE_ONHEAP_HIGHER_MARK,/* happens due to higher mark breach of onheap
> memstore settings
> An offheap memstore can even breach the
> onheap_higher_mark*/
> ABOVE_OFFHEAP_LOWER_MARK,/* happens due to lower mark breach of offheap
> memstore settings*/
> ABOVE_OFFHEAP_HIGHER_MARK;
> {code}
> -> regionServerAccounting does all the accounting.
> -> HeapMemoryTuner is what is litte tricky here. First thing to note is that
> at no point it will try to increase or decrease the
> global.offheap.memstore.size. If there is a heap pressure then it will try to
> increase the memstore heap limit.
> In case of offheap memstore there is always a chance that the heap pressure
> does not increase. In that case we could ideally decrease the heap limit for
> memstore. The current logic of heapmemory tuner is such that things will
> naturally settle down. But on discussion what we thought is let us include
> the flush count that happens due to offheap pressure but give that a lesser
> weightage and thus ensure that the initial decrease on memstore heap limit
> does not happen. Currently that fraction is set as 0.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-17190) eclipse compile master error
[ https://issues.apache.org/jira/browse/HBASE-17190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704379#comment-15704379 ] Jian Yi edited comment on HBASE-17190 at 11/29/16 6:29 AM: --- Thanks! Windows 7中文专业版 jdk1.8.0_111/jre1.8.0_111 Eclipse Platform Version: Neon.1 (4.6.1) Build id: M20160907-1200 Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00) was (Author: eyj...@gmail.com): Thanks! Eclipse Platform Version: Neon.1 (4.6.1) Build id: M20160907-1200 Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00) > eclipse compile master error > > > Key: HBASE-17190 > URL: https://issues.apache.org/jira/browse/HBASE-17190 > Project: HBase > Issue Type: Bug > Components: hbase > Environment: windows 7 + eclipse + maven3.3.9 >Reporter: Jian Yi >Priority: Minor > > [WARNING] The POM for org.apache.maven.shared:maven-shared-utils:jar:0.1 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for > org.apache.maven.shared:maven-shared-incremental:jar:1.1 is invalid, > transitive dependencies (if any) will not be available, enable debug logging > for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-api:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-manager:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-javac:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] Error injecting: org.apache.maven.plugin.compiler.CompilerMojo > java.lang.NoClassDefFoundError: > org/codehaus/plexus/compiler/util/scan/InclusionScanException > at java.lang.Class.getDeclaredConstructors0(Native Method) > at java.lang.Class.privateGetDeclaredConstructors(Unknown Source) > at java.lang.Class.getDeclaredConstructors(Unknown Source) > at > com.google.inject.spi.InjectionPoint.forConstructorOf(InjectionPoint.java:245) > at > com.google.inject.internal.ConstructorBindingImpl.create(ConstructorBindingImpl.java:99) > at > com.google.inject.internal.InjectorImpl.createUninitializedBinding(InjectorImpl.java:658) > at > com.google.inject.internal.InjectorImpl.createJustInTimeBinding(InjectorImpl.java:882) > at > com.google.inject.internal.InjectorImpl.createJustInTimeBindingRecursive(InjectorImpl.java:805) > at > com.google.inject.internal.InjectorImpl.getJustInTimeBinding(InjectorImpl.java:282) > at > com.google.inject.internal.InjectorImpl.getBindingOrThrow(InjectorImpl.java:214) > at > com.google.inject.internal.InjectorImpl.getProviderOrThrow(InjectorImpl.java:1006) > at > com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1038) > at > com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1001) > at > com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) > at > org.eclipse.sisu.space.AbstractDeferredClass.get(AbstractDeferredClass.java:48) > at > com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) > at > com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) > at > com.google.inject.internal.ProviderInternalFactory$1.call(ProviderInternalFactory.java:65) > at > com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:115) > at > org.eclipse.sisu.bean.BeanScheduler$Activator.onProvision(BeanScheduler.java:176) > at > com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:126) > at > com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:68) > at > com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:63) > at > com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) > at > com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) > at > com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) > at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) > at org.eclipse.sisu.inject.Guice4$1.get(Guice4.java:162) > at
[jira] [Commented] (HBASE-17190) eclipse compile master error
[ https://issues.apache.org/jira/browse/HBASE-17190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704379#comment-15704379 ] Jian Yi commented on HBASE-17190: - Thanks! Eclipse Platform Version: Neon.1 (4.6.1) Build id: M20160907-1200 Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00) > eclipse compile master error > > > Key: HBASE-17190 > URL: https://issues.apache.org/jira/browse/HBASE-17190 > Project: HBase > Issue Type: Bug > Components: hbase > Environment: windows 7 + eclipse + maven3.3.9 >Reporter: Jian Yi >Priority: Minor > > [WARNING] The POM for org.apache.maven.shared:maven-shared-utils:jar:0.1 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for > org.apache.maven.shared:maven-shared-incremental:jar:1.1 is invalid, > transitive dependencies (if any) will not be available, enable debug logging > for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-api:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-manager:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-javac:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] Error injecting: org.apache.maven.plugin.compiler.CompilerMojo > java.lang.NoClassDefFoundError: > org/codehaus/plexus/compiler/util/scan/InclusionScanException > at java.lang.Class.getDeclaredConstructors0(Native Method) > at java.lang.Class.privateGetDeclaredConstructors(Unknown Source) > at java.lang.Class.getDeclaredConstructors(Unknown Source) > at > com.google.inject.spi.InjectionPoint.forConstructorOf(InjectionPoint.java:245) > at > com.google.inject.internal.ConstructorBindingImpl.create(ConstructorBindingImpl.java:99) > at > com.google.inject.internal.InjectorImpl.createUninitializedBinding(InjectorImpl.java:658) > at > com.google.inject.internal.InjectorImpl.createJustInTimeBinding(InjectorImpl.java:882) > at > com.google.inject.internal.InjectorImpl.createJustInTimeBindingRecursive(InjectorImpl.java:805) > at > com.google.inject.internal.InjectorImpl.getJustInTimeBinding(InjectorImpl.java:282) > at > com.google.inject.internal.InjectorImpl.getBindingOrThrow(InjectorImpl.java:214) > at > com.google.inject.internal.InjectorImpl.getProviderOrThrow(InjectorImpl.java:1006) > at > com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1038) > at > com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1001) > at > com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) > at > org.eclipse.sisu.space.AbstractDeferredClass.get(AbstractDeferredClass.java:48) > at > com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) > at > com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) > at > com.google.inject.internal.ProviderInternalFactory$1.call(ProviderInternalFactory.java:65) > at > com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:115) > at > org.eclipse.sisu.bean.BeanScheduler$Activator.onProvision(BeanScheduler.java:176) > at > com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:126) > at > com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:68) > at > com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:63) > at > com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) > at > com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) > at > com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) > at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) > at org.eclipse.sisu.inject.Guice4$1.get(Guice4.java:162) > at org.eclipse.sisu.inject.LazyBeanEntry.getValue(LazyBeanEntry.java:81) > at > org.eclipse.sisu.plexus.LazyPlexusBean.getValue(LazyPlexusBean.java:51) > at > org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:263) > at >
[jira] [Commented] (HBASE-17167) Pass mvcc to client when scan
[
https://issues.apache.org/jira/browse/HBASE-17167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704375#comment-15704375
]
Hadoop QA commented on HBASE-17167:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 28s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 28s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 50s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
40s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 13s
{color} | {color:red} hbase-protocol-shaded in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 56s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 56s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 56s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 9s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 23s
{color} | {color:red} Patch generated 1 new protoc errors in hbase-server.
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 14s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s
{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 35s
{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 6s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 109m 59s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
22s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 187m 20s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12840788/HBASE-17167-v4.patch |
| JIRA Issue | HBASE-17167 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile cc hbaseprotoc |
| uname | Linux 05f8e23d6124 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016
[jira] [Commented] (HBASE-17086) Add comments to explain why Cell#getTagsLength() returns an int, rather than a short
[ https://issues.apache.org/jira/browse/HBASE-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704337#comment-15704337 ] Anoop Sam John commented on HBASE-17086: Better NOT to mention KV and KV format in Cell.. What we need to say is that HBase uses 2 bytes to store cell's tag length. As it is going to be +ve numbers always, we save sign bit and so max tags length = 2 * short.MAX +1 Tell the number. So even if the return type is int, the max tags length is less than Int.MAX value. > Add comments to explain why Cell#getTagsLength() returns an int, rather than > a short > > > Key: HBASE-17086 > URL: https://issues.apache.org/jira/browse/HBASE-17086 > Project: HBase > Issue Type: Improvement > Components: Interface >Affects Versions: 2.0.0 >Reporter: Xiang Li >Assignee: Xiang Li >Priority: Minor > Attachments: HBASE-17086.master.000.patch > > > In the Cell interface, getTagsLength() returns a int > But in the KeyValue implementation, tags length is of 2 bytes. Also in > ExtendedCell, when explaining the KeyValue format, tags length is stated to > be 2 bytes > Any plan to update Cell interface to make getTagsLength() returns a short ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17167) Pass mvcc to client when scan
[ https://issues.apache.org/jira/browse/HBASE-17167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704333#comment-15704333 ] Anoop Sam John commented on HBASE-17167: +1 > Pass mvcc to client when scan > - > > Key: HBASE-17167 > URL: https://issues.apache.org/jira/browse/HBASE-17167 > Project: HBase > Issue Type: Sub-task > Components: Client, scan >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17167-v1.patch, HBASE-17167-v2.patch, > HBASE-17167-v3.patch, HBASE-17167-v4.patch, HBASE-17167.patch > > > For the current implementation, if we use batch or allowPartial when scan, > then the row level atomic can not be guaranteed if we need to restart a scan > in the middle of a record due to region move or something else. > We can return the mvcc used to open scanner to client and client could use > this mvcc to restart a scan to get row level atomic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16561) Add metrics about read/write/scan queue length and active read/write/scan handler count
[ https://issues.apache.org/jira/browse/HBASE-16561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704294#comment-15704294 ] Guanghao Zhang commented on HBASE-16561: Failed ut not related and passed locally. > Add metrics about read/write/scan queue length and active read/write/scan > handler count > --- > > Key: HBASE-16561 > URL: https://issues.apache.org/jira/browse/HBASE-16561 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, metrics >Affects Versions: 2.0.0, 1.4.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16561-branch-1.patch, HBASE-16561-v1.patch, > HBASE-16561.patch > > > Now there are only metrics about total queue length and active rpc handler > count. But in the RWQueueRpcExecutor, there are different queues and handlers > for read/write/scan request. I thought it is necessary to add more metrics > for RWQueueRpcExecutor. When use it in production cluster, we can adjust the > config of queues and handlers according to the metrics. > Review url: https://reviews.apache.org/r/54072/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17190) eclipse compile master error
[ https://issues.apache.org/jira/browse/HBASE-17190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704293#comment-15704293 ] Duo Zhang commented on HBASE-17190: --- OK. Let me have a try for windows... > eclipse compile master error > > > Key: HBASE-17190 > URL: https://issues.apache.org/jira/browse/HBASE-17190 > Project: HBase > Issue Type: Bug > Components: hbase > Environment: windows 7 + eclipse + maven3.3.9 >Reporter: Jian Yi >Priority: Minor > > [WARNING] The POM for org.apache.maven.shared:maven-shared-utils:jar:0.1 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for > org.apache.maven.shared:maven-shared-incremental:jar:1.1 is invalid, > transitive dependencies (if any) will not be available, enable debug logging > for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-api:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-manager:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-javac:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] Error injecting: org.apache.maven.plugin.compiler.CompilerMojo > java.lang.NoClassDefFoundError: > org/codehaus/plexus/compiler/util/scan/InclusionScanException > at java.lang.Class.getDeclaredConstructors0(Native Method) > at java.lang.Class.privateGetDeclaredConstructors(Unknown Source) > at java.lang.Class.getDeclaredConstructors(Unknown Source) > at > com.google.inject.spi.InjectionPoint.forConstructorOf(InjectionPoint.java:245) > at > com.google.inject.internal.ConstructorBindingImpl.create(ConstructorBindingImpl.java:99) > at > com.google.inject.internal.InjectorImpl.createUninitializedBinding(InjectorImpl.java:658) > at > com.google.inject.internal.InjectorImpl.createJustInTimeBinding(InjectorImpl.java:882) > at > com.google.inject.internal.InjectorImpl.createJustInTimeBindingRecursive(InjectorImpl.java:805) > at > com.google.inject.internal.InjectorImpl.getJustInTimeBinding(InjectorImpl.java:282) > at > com.google.inject.internal.InjectorImpl.getBindingOrThrow(InjectorImpl.java:214) > at > com.google.inject.internal.InjectorImpl.getProviderOrThrow(InjectorImpl.java:1006) > at > com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1038) > at > com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1001) > at > com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) > at > org.eclipse.sisu.space.AbstractDeferredClass.get(AbstractDeferredClass.java:48) > at > com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) > at > com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) > at > com.google.inject.internal.ProviderInternalFactory$1.call(ProviderInternalFactory.java:65) > at > com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:115) > at > org.eclipse.sisu.bean.BeanScheduler$Activator.onProvision(BeanScheduler.java:176) > at > com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:126) > at > com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:68) > at > com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:63) > at > com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) > at > com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) > at > com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) > at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) > at org.eclipse.sisu.inject.Guice4$1.get(Guice4.java:162) > at org.eclipse.sisu.inject.LazyBeanEntry.getValue(LazyBeanEntry.java:81) > at > org.eclipse.sisu.plexus.LazyPlexusBean.getValue(LazyPlexusBean.java:51) > at > org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:263) > at > org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:255) > at >
[jira] [Commented] (HBASE-17110) Add an "Overall Strategy" option(balanced both on table level and server level) to SimpleLoadBalancer
[
https://issues.apache.org/jira/browse/HBASE-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704277#comment-15704277
]
Hadoop QA commented on HBASE-17110:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
6s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 44s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 20s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 109m 20s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 26s
{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 156m 22s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush |
| Timed out junit tests |
org.apache.hadoop.hbase.namespace.TestNamespaceAuditor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12840783/HBASE-17110-V8.patch |
| JIRA Issue | HBASE-17110 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux ccb20ecba7b2 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / cc03f7a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/4665/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
[jira] [Commented] (HBASE-17181) Let HBase thrift2 support TThreadedSelectorServer
[ https://issues.apache.org/jira/browse/HBASE-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704271#comment-15704271 ] stack commented on HBASE-17181: --- Can you post a difference from master, a patch, rather than files so we can see what has changed [~eyjian]? This section from the hbase refguide tries to lead you through how (there is some nice tooling to help you): http://hbase.apache.org/book.html#submitting.patches.create. If you want to learn more about how hbase does formatting, etc., there is http://hbase.apache.org/book.html#developer. Thanks [~eyjian] > Let HBase thrift2 support TThreadedSelectorServer > - > > Key: HBASE-17181 > URL: https://issues.apache.org/jira/browse/HBASE-17181 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 1.2.3 >Reporter: Jian Yi >Priority: Minor > Labels: features > Fix For: 1.2.3 > > Attachments: HBASE-17181-V1.patch, HBASE-17181-V2.patch, > HBASE-17181-V3.patch, ThriftServer.java > > Original Estimate: 2h > Remaining Estimate: 2h > > Add TThreadedSelectorServer for HBase Thrift2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14882) Provide a Put API that adds the provided family, qualifier, value without copying
[
https://issues.apache.org/jira/browse/HBASE-14882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-14882:
-
Status: Open (was: Patch Available)
> Provide a Put API that adds the provided family, qualifier, value without
> copying
> -
>
> Key: HBASE-14882
> URL: https://issues.apache.org/jira/browse/HBASE-14882
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.0
>Reporter: Jerry He
>Assignee: Xiang Li
> Fix For: 2.0.0
>
> Attachments: HBASE-14882.master.000.patch,
> HBASE-14882.master.001.patch, HBASE-14882.master.002.patch,
> HBASE-14882.master.003.patch
>
>
> In the Put API, we have addImmutable()
> {code}
> /**
>* See {@link #addColumn(byte[], byte[], byte[])}. This version expects
>* that the underlying arrays won't change. It's intended
>* for usage internal HBase to and for advanced client applications.
>*/
> public Put addImmutable(byte [] family, byte [] qualifier, byte [] value)
> {code}
> But in the implementation, the family, qualifier and value are still being
> copied locally to create kv.
> Hopefully we should provide an API that truly uses immutable family,
> qualifier and value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17181) Let HBase thrift2 support TThreadedSelectorServer
[ https://issues.apache.org/jira/browse/HBASE-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704252#comment-15704252 ] stack commented on HBASE-17181: --- Just to say there's no harm in us getting used to handling chinese ideograms. Its good for us over on this end of the world. > Let HBase thrift2 support TThreadedSelectorServer > - > > Key: HBASE-17181 > URL: https://issues.apache.org/jira/browse/HBASE-17181 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 1.2.3 >Reporter: Jian Yi >Priority: Minor > Labels: features > Fix For: 1.2.3 > > Attachments: HBASE-17181-V1.patch, HBASE-17181-V2.patch, > HBASE-17181-V3.patch, ThriftServer.java > > Original Estimate: 2h > Remaining Estimate: 2h > > Add TThreadedSelectorServer for HBase Thrift2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17187) DoNotRetryExceptions from coprocessors should bubble up to the application
[
https://issues.apache.org/jira/browse/HBASE-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704244#comment-15704244
]
Hadoop QA commented on HBASE-17187:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 46s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 105m 6s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 151m 16s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12840777/hbase-17187_v1.patch |
| JIRA Issue | HBASE-17187 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 9386de39dc64 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cc03f7a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4664/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4664/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> DoNotRetryExceptions from coprocessors should
[jira] [Commented] (HBASE-17048) Calcuate suitable ByteBuf size when allocating send buffer in FanOutOneBlockAsyncDFSOutput
[ https://issues.apache.org/jira/browse/HBASE-17048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704238#comment-15704238 ] stack commented on HBASE-17048: --- Looks good to me. Nice. > Calcuate suitable ByteBuf size when allocating send buffer in > FanOutOneBlockAsyncDFSOutput > -- > > Key: HBASE-17048 > URL: https://issues.apache.org/jira/browse/HBASE-17048 > Project: HBase > Issue Type: Sub-task > Components: wal >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-17012_3.patch, HBASE-17048.patch, > HBASE-17048_1.patch, HBASE-17048_3.patch > > > As [~ram_krish] mentioned in HBASE-17021 > https://issues.apache.org/jira/browse/HBASE-17021?focusedCommentId=15646938=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15646938 > The default ByteBuf size is 256B which is too small. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17128) Find Cause of a Write Perf Regression in branch-1.2
[
https://issues.apache.org/jira/browse/HBASE-17128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704202#comment-15704202
]
stack commented on HBASE-17128:
---
There is also HBASE-17072 [~gbaecher] You able to patch your hbase?
> Find Cause of a Write Perf Regression in branch-1.2
> ---
>
> Key: HBASE-17128
> URL: https://issues.apache.org/jira/browse/HBASE-17128
> Project: HBase
> Issue Type: Task
>Reporter: stack
>
> As reported by [~gbaecher] up on the mailing list, there is a regression in
> 1.2. The regression is in a CDH version of 1.2 actually but the CDH hbase is
> a near pure 1.2. This is a working issue to figure which of the below changes
> brought on slower writes (The list comes from doing the following...git log
> --oneline
> remotes/origin/cdh5-1.2.0_5.8.0_dev..remotes/origin/cdh5-1.2.0_5.9.0_dev ...
> I stripped the few CDH specific changes, packaging and tagging only, and then
> made two groupings; candidates and the unlikelies):
> {code}
> 1 bbc6762 HBASE-16023 Fastpath for the FIFO rpcscheduler Adds an executor
> that does balanced queue and fast path handing off requests directly to
> waiting handlers if any present. Idea taken from Apace Kudu (incubating). See
> https://gerr#
> 2 a260917 HBASE-16288 HFile intermediate block level indexes might recurse
> forever creating multi TB files
> 3 5633281 HBASE-15811 Batch Get after batch Put does not fetch all Cells We
> were not waiting on all executors in a batch to complete. The test for
> no-more-executors was damaged by the 0.99/0.98.4 fix "HBASE-11403 Fix race
> conditions aro#
> 4 780f720 HBASE-11625 - Verifies data before building HFileBlock. - Adds
> HFileBlock.Header class which contains information about location of fields.
> Testing: Adds CorruptedFSReaderImpl to TestChecksum. (Apekshit)
> 5 d735680 HBASE-12133 Add FastLongHistogram for metric computation (Yi Deng)
> 6 c4ee832 HBASE-15222 Use less contended classes for metrics
> 7
> 8 17320a4 HBASE-15683 Min latency in latency histograms are emitted as
> Long.MAX_VALUE
> 9 283b39f HBASE-15396 Enhance mapreduce.TableSplit to add encoded region
> name
> 10 39db592 HBASE-16195 Should not add chunk into chunkQueue if not using
> chunk pool in HeapMemStoreLAB
> 11 5ff28b7 HBASE-16194 Should count in MSLAB chunk allocation into heap size
> change when adding duplicate cells
> 12 5e3e0d2 HBASE-16318 fail build while rendering velocity template if
> dependency license isn't in whitelist.
> 13 3ed66e3 HBASE-16318 consistently use the correct name for 'Apache
> License, Version 2.0'
> 14 351832d HBASE-16340 exclude Xerces iplementation jars from coming in
> transitively.
> 15 b6aa4be HBASE-16321 ensure no findbugs-jsr305
> 16 4f9dde7 HBASE-16317 revert all ESAPI changes
> 17 71b6a8a HBASE-16284 Unauthorized client can shutdown the cluster (Deokwoo
> Han)
> 18 523753f HBASE-16450 Shell tool to dump replication queues
> 19 ca5f2ee HBASE-16379 [replication] Minor improvement to
> replication/copy_tables_desc.rb
> 20 effd105 HBASE-16135 PeerClusterZnode under rs of removed peer may never
> be deleted
> 21 a5c6610 HBASE-16319 Fix TestCacheOnWrite after HBASE-16288
> 22 1956bb0 HBASE-15808 Reduce potential bulk load intermediate space usage
> and waste
> 23 031c54e HBASE-16096 Backport. Cleanly remove replication peers from
> ZooKeeper.
> 24 60a3b12 HBASE-14963 Remove use of Guava Stopwatch from HBase client code
> (Devaraj Das)
> 25 c7724fc HBASE-16207 can't restore snapshot without "Admin" permission
> 26 8322a0b HBASE-16227 [Shell] Column value formatter not working in scans.
> Tested : manually using shell.
> 27 8f86658 HBASE-14818 user_permission does not list namespace permissions
> (li xiang)
> 28 775cd21 HBASE-15465 userPermission returned by getUserPermission() for
> the selected namespace does not have namespace set (li xiang)
> 29 8d85aff HBASE-16093 Fix splits failed before creating daughter regions
> leave meta inconsistent
> 30 bc41317 HBASE-16140 bump owasp.esapi from 2.1.0 to 2.1.0.1
> 31 6fc70cd HBASE-16035 Nested AutoCloseables might not all get closed (Sean
> Mackrory)
> 32 fe28fe84 HBASE-15891. Closeable resources potentially not getting closed
> if exception is thrown.
> 33 1d2bf3c HBASE-14644 Region in transition metric is broken -- addendum
> (Huaxiang Sun)
> 34 fd5f56c HBASE-16056 Procedure v2 - fix master crash for FileNotFound
> 35 10cd038 HBASE-16034 Fix ProcedureTestingUtility#LoadCounter.setMaxProcId()
> 36 dae4db4 HBASE-15872 Split TestWALProcedureStore
> 37 e638d86 HBASE-14644 Region in transition metric is broken (Huaxiang Sun)
> 38 f01b01d HBASE-15496 Throw RowTooBigException only for user scan/get
> (Guanghao Zhang)
> 39 cc0ce66 HBASE-15746 Remove extra RegionCoprocessor preClose() in
>
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16261: - Attachment: HBASE-16261-V4.patch > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch, > HBASE-16261-V3.patch, HBASE-16261-V4.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17181) Let HBase thrift2 support TThreadedSelectorServer
[ https://issues.apache.org/jira/browse/HBASE-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Yi updated HBASE-17181: Attachment: ThriftServer.java source code file based on master > Let HBase thrift2 support TThreadedSelectorServer > - > > Key: HBASE-17181 > URL: https://issues.apache.org/jira/browse/HBASE-17181 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 1.2.3 >Reporter: Jian Yi >Priority: Minor > Labels: features > Fix For: 1.2.3 > > Attachments: HBASE-17181-V1.patch, HBASE-17181-V2.patch, > HBASE-17181-V3.patch, ThriftServer.java > > Original Estimate: 2h > Remaining Estimate: 2h > > Add TThreadedSelectorServer for HBase Thrift2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16561) Add metrics about read/write/scan queue length and active read/write/scan handler count
[
https://issues.apache.org/jira/browse/HBASE-16561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704137#comment-15704137
]
Hadoop QA commented on HBASE-16561:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m 41s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 16s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
3s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
47s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 44s
{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 44s
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
33s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 51s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 16s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 20s
{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 82m 33s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
50s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 134m 37s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestEncryptionKeyRotation |
\\
[jira] [Commented] (HBASE-17114) Add an option to set special retry pause when encountering CallQueueTooBigException
[
https://issues.apache.org/jira/browse/HBASE-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704136#comment-15704136
]
Yu Li commented on HBASE-17114:
---
Thanks for review [~ghelmling], will add the config/description in
hbase-default.xml.
Regarding the test, let's leave it as is for now and give it a fix if turned
out to be flaky on ASF infra. Will keep an eye on the post commit tests.
> Add an option to set special retry pause when encountering
> CallQueueTooBigException
> ---
>
> Key: HBASE-17114
> URL: https://issues.apache.org/jira/browse/HBASE-17114
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-17114.patch, HBASE-17114.v2.patch
>
>
> As titled, after HBASE-15146 we will throw {{CallQueueTooBigException}}
> instead of dead-wait. This is good for performance for most cases but might
> cause a side-effect that if too many clients connect to the busy RS, that the
> retry requests may come over and over again and RS never got the chance for
> recovering, and the issue will become especially critical when the target
> region is META.
> So here in this JIRA we propose to supply some special retry pause for CQTBE
> in name of {{hbase.client.pause.special}}, and by default it will be 500ms (5
> times of {{hbase.client.pause}} default value)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-17190) eclipse compile master error
[ https://issues.apache.org/jira/browse/HBASE-17190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704134#comment-15704134 ] Jian Yi edited comment on HBASE-17190 at 11/29/16 4:22 AM: --- eclise with maven 3.3.9, and goals is: clean install -DskipTests. hbase-1.2.3 is ok. was (Author: eyj...@gmail.com): eclise with maven 3.3.9, and goals is: clean install -DskipTests > eclipse compile master error > > > Key: HBASE-17190 > URL: https://issues.apache.org/jira/browse/HBASE-17190 > Project: HBase > Issue Type: Bug > Components: hbase > Environment: windows 7 + eclipse + maven3.3.9 >Reporter: Jian Yi >Priority: Minor > > [WARNING] The POM for org.apache.maven.shared:maven-shared-utils:jar:0.1 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for > org.apache.maven.shared:maven-shared-incremental:jar:1.1 is invalid, > transitive dependencies (if any) will not be available, enable debug logging > for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-api:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-manager:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-javac:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] Error injecting: org.apache.maven.plugin.compiler.CompilerMojo > java.lang.NoClassDefFoundError: > org/codehaus/plexus/compiler/util/scan/InclusionScanException > at java.lang.Class.getDeclaredConstructors0(Native Method) > at java.lang.Class.privateGetDeclaredConstructors(Unknown Source) > at java.lang.Class.getDeclaredConstructors(Unknown Source) > at > com.google.inject.spi.InjectionPoint.forConstructorOf(InjectionPoint.java:245) > at > com.google.inject.internal.ConstructorBindingImpl.create(ConstructorBindingImpl.java:99) > at > com.google.inject.internal.InjectorImpl.createUninitializedBinding(InjectorImpl.java:658) > at > com.google.inject.internal.InjectorImpl.createJustInTimeBinding(InjectorImpl.java:882) > at > com.google.inject.internal.InjectorImpl.createJustInTimeBindingRecursive(InjectorImpl.java:805) > at > com.google.inject.internal.InjectorImpl.getJustInTimeBinding(InjectorImpl.java:282) > at > com.google.inject.internal.InjectorImpl.getBindingOrThrow(InjectorImpl.java:214) > at > com.google.inject.internal.InjectorImpl.getProviderOrThrow(InjectorImpl.java:1006) > at > com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1038) > at > com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1001) > at > com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) > at > org.eclipse.sisu.space.AbstractDeferredClass.get(AbstractDeferredClass.java:48) > at > com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) > at > com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) > at > com.google.inject.internal.ProviderInternalFactory$1.call(ProviderInternalFactory.java:65) > at > com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:115) > at > org.eclipse.sisu.bean.BeanScheduler$Activator.onProvision(BeanScheduler.java:176) > at > com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:126) > at > com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:68) > at > com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:63) > at > com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) > at > com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) > at > com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) > at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) > at org.eclipse.sisu.inject.Guice4$1.get(Guice4.java:162) > at org.eclipse.sisu.inject.LazyBeanEntry.getValue(LazyBeanEntry.java:81) > at > org.eclipse.sisu.plexus.LazyPlexusBean.getValue(LazyPlexusBean.java:51) > at > org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:263) >
[jira] [Commented] (HBASE-17190) eclipse compile master error
[ https://issues.apache.org/jira/browse/HBASE-17190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704134#comment-15704134 ] Jian Yi commented on HBASE-17190: - eclise with maven 3.3.9, and goals is: clean install -DskipTests > eclipse compile master error > > > Key: HBASE-17190 > URL: https://issues.apache.org/jira/browse/HBASE-17190 > Project: HBase > Issue Type: Bug > Components: hbase > Environment: windows 7 + eclipse + maven3.3.9 >Reporter: Jian Yi >Priority: Minor > > [WARNING] The POM for org.apache.maven.shared:maven-shared-utils:jar:0.1 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for > org.apache.maven.shared:maven-shared-incremental:jar:1.1 is invalid, > transitive dependencies (if any) will not be available, enable debug logging > for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-api:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-manager:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-javac:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] Error injecting: org.apache.maven.plugin.compiler.CompilerMojo > java.lang.NoClassDefFoundError: > org/codehaus/plexus/compiler/util/scan/InclusionScanException > at java.lang.Class.getDeclaredConstructors0(Native Method) > at java.lang.Class.privateGetDeclaredConstructors(Unknown Source) > at java.lang.Class.getDeclaredConstructors(Unknown Source) > at > com.google.inject.spi.InjectionPoint.forConstructorOf(InjectionPoint.java:245) > at > com.google.inject.internal.ConstructorBindingImpl.create(ConstructorBindingImpl.java:99) > at > com.google.inject.internal.InjectorImpl.createUninitializedBinding(InjectorImpl.java:658) > at > com.google.inject.internal.InjectorImpl.createJustInTimeBinding(InjectorImpl.java:882) > at > com.google.inject.internal.InjectorImpl.createJustInTimeBindingRecursive(InjectorImpl.java:805) > at > com.google.inject.internal.InjectorImpl.getJustInTimeBinding(InjectorImpl.java:282) > at > com.google.inject.internal.InjectorImpl.getBindingOrThrow(InjectorImpl.java:214) > at > com.google.inject.internal.InjectorImpl.getProviderOrThrow(InjectorImpl.java:1006) > at > com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1038) > at > com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1001) > at > com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) > at > org.eclipse.sisu.space.AbstractDeferredClass.get(AbstractDeferredClass.java:48) > at > com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) > at > com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) > at > com.google.inject.internal.ProviderInternalFactory$1.call(ProviderInternalFactory.java:65) > at > com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:115) > at > org.eclipse.sisu.bean.BeanScheduler$Activator.onProvision(BeanScheduler.java:176) > at > com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:126) > at > com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:68) > at > com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:63) > at > com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) > at > com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) > at > com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) > at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) > at org.eclipse.sisu.inject.Guice4$1.get(Guice4.java:162) > at org.eclipse.sisu.inject.LazyBeanEntry.getValue(LazyBeanEntry.java:81) > at > org.eclipse.sisu.plexus.LazyPlexusBean.getValue(LazyPlexusBean.java:51) > at > org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:263) > at > org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:255) > at >
[jira] [Commented] (HBASE-16755) Honor flush policy under global memstore pressure
[
https://issues.apache.org/jira/browse/HBASE-16755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704105#comment-15704105
]
Hadoop QA commented on HBASE-16755:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
2s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 57s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 88m 11s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 127m 18s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12840774/HBASE-16755.v0.patch |
| JIRA Issue | HBASE-16755 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux c19a1aa82431 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / cc03f7a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4661/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4661/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Honor flush policy under global memstore pressure
> -
>
> Key: HBASE-16755
> URL: https://issues.apache.org/jira/browse/HBASE-16755
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Ashu Pachauri
>
[jira] [Commented] (HBASE-17190) eclipse compile master error
[ https://issues.apache.org/jira/browse/HBASE-17190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704091#comment-15704091 ] Duo Zhang commented on HBASE-17190: --- What command do you use to compile HBase? Have you trid 'mvn clean install -DskipTests'? > eclipse compile master error > > > Key: HBASE-17190 > URL: https://issues.apache.org/jira/browse/HBASE-17190 > Project: HBase > Issue Type: Bug > Components: hbase > Environment: windows 7 + eclipse + maven3.3.9 >Reporter: Jian Yi >Priority: Minor > > [WARNING] The POM for org.apache.maven.shared:maven-shared-utils:jar:0.1 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for > org.apache.maven.shared:maven-shared-incremental:jar:1.1 is invalid, > transitive dependencies (if any) will not be available, enable debug logging > for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-api:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-manager:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] The POM for org.codehaus.plexus:plexus-compiler-javac:jar:2.4 is > invalid, transitive dependencies (if any) will not be available, enable debug > logging for more details > [WARNING] Error injecting: org.apache.maven.plugin.compiler.CompilerMojo > java.lang.NoClassDefFoundError: > org/codehaus/plexus/compiler/util/scan/InclusionScanException > at java.lang.Class.getDeclaredConstructors0(Native Method) > at java.lang.Class.privateGetDeclaredConstructors(Unknown Source) > at java.lang.Class.getDeclaredConstructors(Unknown Source) > at > com.google.inject.spi.InjectionPoint.forConstructorOf(InjectionPoint.java:245) > at > com.google.inject.internal.ConstructorBindingImpl.create(ConstructorBindingImpl.java:99) > at > com.google.inject.internal.InjectorImpl.createUninitializedBinding(InjectorImpl.java:658) > at > com.google.inject.internal.InjectorImpl.createJustInTimeBinding(InjectorImpl.java:882) > at > com.google.inject.internal.InjectorImpl.createJustInTimeBindingRecursive(InjectorImpl.java:805) > at > com.google.inject.internal.InjectorImpl.getJustInTimeBinding(InjectorImpl.java:282) > at > com.google.inject.internal.InjectorImpl.getBindingOrThrow(InjectorImpl.java:214) > at > com.google.inject.internal.InjectorImpl.getProviderOrThrow(InjectorImpl.java:1006) > at > com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1038) > at > com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1001) > at > com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) > at > org.eclipse.sisu.space.AbstractDeferredClass.get(AbstractDeferredClass.java:48) > at > com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) > at > com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) > at > com.google.inject.internal.ProviderInternalFactory$1.call(ProviderInternalFactory.java:65) > at > com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:115) > at > org.eclipse.sisu.bean.BeanScheduler$Activator.onProvision(BeanScheduler.java:176) > at > com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:126) > at > com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:68) > at > com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:63) > at > com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) > at > com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) > at > com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) > at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) > at org.eclipse.sisu.inject.Guice4$1.get(Guice4.java:162) > at org.eclipse.sisu.inject.LazyBeanEntry.getValue(LazyBeanEntry.java:81) > at > org.eclipse.sisu.plexus.LazyPlexusBean.getValue(LazyPlexusBean.java:51) > at > org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:263) > at > org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:255) > at >
[jira] [Created] (HBASE-17190) eclipse compile master error
Jian Yi created HBASE-17190: --- Summary: eclipse compile master error Key: HBASE-17190 URL: https://issues.apache.org/jira/browse/HBASE-17190 Project: HBase Issue Type: Bug Components: hbase Environment: windows 7 + eclipse + maven3.3.9 Reporter: Jian Yi Priority: Minor [WARNING] The POM for org.apache.maven.shared:maven-shared-utils:jar:0.1 is invalid, transitive dependencies (if any) will not be available, enable debug logging for more details [WARNING] The POM for org.apache.maven.shared:maven-shared-incremental:jar:1.1 is invalid, transitive dependencies (if any) will not be available, enable debug logging for more details [WARNING] The POM for org.codehaus.plexus:plexus-compiler-api:jar:2.4 is invalid, transitive dependencies (if any) will not be available, enable debug logging for more details [WARNING] The POM for org.codehaus.plexus:plexus-compiler-manager:jar:2.4 is invalid, transitive dependencies (if any) will not be available, enable debug logging for more details [WARNING] The POM for org.codehaus.plexus:plexus-compiler-javac:jar:2.4 is invalid, transitive dependencies (if any) will not be available, enable debug logging for more details [WARNING] Error injecting: org.apache.maven.plugin.compiler.CompilerMojo java.lang.NoClassDefFoundError: org/codehaus/plexus/compiler/util/scan/InclusionScanException at java.lang.Class.getDeclaredConstructors0(Native Method) at java.lang.Class.privateGetDeclaredConstructors(Unknown Source) at java.lang.Class.getDeclaredConstructors(Unknown Source) at com.google.inject.spi.InjectionPoint.forConstructorOf(InjectionPoint.java:245) at com.google.inject.internal.ConstructorBindingImpl.create(ConstructorBindingImpl.java:99) at com.google.inject.internal.InjectorImpl.createUninitializedBinding(InjectorImpl.java:658) at com.google.inject.internal.InjectorImpl.createJustInTimeBinding(InjectorImpl.java:882) at com.google.inject.internal.InjectorImpl.createJustInTimeBindingRecursive(InjectorImpl.java:805) at com.google.inject.internal.InjectorImpl.getJustInTimeBinding(InjectorImpl.java:282) at com.google.inject.internal.InjectorImpl.getBindingOrThrow(InjectorImpl.java:214) at com.google.inject.internal.InjectorImpl.getProviderOrThrow(InjectorImpl.java:1006) at com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1038) at com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1001) at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) at org.eclipse.sisu.space.AbstractDeferredClass.get(AbstractDeferredClass.java:48) at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) at com.google.inject.internal.ProviderInternalFactory$1.call(ProviderInternalFactory.java:65) at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:115) at org.eclipse.sisu.bean.BeanScheduler$Activator.onProvision(BeanScheduler.java:176) at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:126) at com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:68) at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:63) at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) at org.eclipse.sisu.inject.Guice4$1.get(Guice4.java:162) at org.eclipse.sisu.inject.LazyBeanEntry.getValue(LazyBeanEntry.java:81) at org.eclipse.sisu.plexus.LazyPlexusBean.getValue(LazyPlexusBean.java:51) at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:263) at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:255) at org.apache.maven.plugin.internal.DefaultMavenPluginManager.getConfiguredMojo(DefaultMavenPluginManager.java:517) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:121) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
[jira] [Updated] (HBASE-17048) Calcuate suitable ByteBuf size when allocating send buffer in FanOutOneBlockAsyncDFSOutput
[ https://issues.apache.org/jira/browse/HBASE-17048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-17048: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to master. Thanks for the reviews - Duo and Anoop. > Calcuate suitable ByteBuf size when allocating send buffer in > FanOutOneBlockAsyncDFSOutput > -- > > Key: HBASE-17048 > URL: https://issues.apache.org/jira/browse/HBASE-17048 > Project: HBase > Issue Type: Sub-task > Components: wal >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-17012_3.patch, HBASE-17048.patch, > HBASE-17048_1.patch, HBASE-17048_3.patch > > > As [~ram_krish] mentioned in HBASE-17021 > https://issues.apache.org/jira/browse/HBASE-17021?focusedCommentId=15646938=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15646938 > The default ByteBuf size is 256B which is too small. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17048) Calcuate suitable ByteBuf size when allocating send buffer in FanOutOneBlockAsyncDFSOutput
[ https://issues.apache.org/jira/browse/HBASE-17048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-17048: --- Summary: Calcuate suitable ByteBuf size when allocating send buffer in FanOutOneBlockAsyncDFSOutput (was: Introduce a helper class to calcuate suitable ByteBuf size when allocating send buffer in FanOutOneBlockAsyncDFSOutputHelper) > Calcuate suitable ByteBuf size when allocating send buffer in > FanOutOneBlockAsyncDFSOutput > -- > > Key: HBASE-17048 > URL: https://issues.apache.org/jira/browse/HBASE-17048 > Project: HBase > Issue Type: Sub-task > Components: wal >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-17012_3.patch, HBASE-17048.patch, > HBASE-17048_1.patch, HBASE-17048_3.patch > > > As [~ram_krish] mentioned in HBASE-17021 > https://issues.apache.org/jira/browse/HBASE-17021?focusedCommentId=15646938=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15646938 > The default ByteBuf size is 256B which is too small. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17187) DoNotRetryExceptions from coprocessors should bubble up to the application
[
https://issues.apache.org/jira/browse/HBASE-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704068#comment-15704068
]
Josh Elser commented on HBASE-17187:
bq. We are using the default num of retries only in the case where the
exception is thrown is NOT a DNRIOE. Otherwise, we do not retry. The Phoenix
test failure is throwing DNRIOE I think.
Oh superb. My brain was expecting to see something in ClientScanner for that
and ignored the server-side change :)
+1 from me, for what it's worth :P
> DoNotRetryExceptions from coprocessors should bubble up to the application
> --
>
> Key: HBASE-17187
> URL: https://issues.apache.org/jira/browse/HBASE-17187
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-17187_v1.patch
>
>
> In HBASE-16604, we fixed a case where scanner retries was causing the scan to
> miss some data in case the scanner is left with a dirty state (like a
> half-seeked KVHeap).
> The patch introduced a minor compatibility issue, because now if a
> coprocessor throws DNRIOE, we still retry the ClientScanner indefinitely.
> The test {{ServerExceptionIT}} in Phoenix is failing because of this with
> HBASE-16604.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17178) Add region balance throttling
[ https://issues.apache.org/jira/browse/HBASE-17178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704065#comment-15704065 ] Phil Yang commented on HBASE-17178: --- Maybe we need a config to limit the max ratio of RIT regions? From 0.0 to 1.0, 0.0 means only one region. It is a direct way to protect the availability, 0.05 means at least 95% availability. And we will move as slow as possible to prevent RIT regions reach this bottom line. > Add region balance throttling > - > > Key: HBASE-17178 > URL: https://issues.apache.org/jira/browse/HBASE-17178 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17178-v1.patch > > > Our online cluster serves dozens of tables and different tables serve for > different services. If the balancer moves too many regions in the same time, > it will decrease the availability for some table or some services. So we add > region balance throttling on our online serve cluster. > We introduce a new config hbase.balancer.max.balancing.regions, which means > the max number of regions in transition when balancing. > If we config this to 1 and a table have 100 regions, then the table will have > 99 regions available at any time. It helps a lot for our use case and it has > been running a long time > our production cluster. > But for some use case, we need the balancer run faster. If a cluster has 100 > regionservers, then it add 50 new regionservers for peak requests. Then it > need balancer run as soon as > possible and let the cluster reach a balance state soon. Our idea is compute > max number of regions in transition by the max balancing time and the average > time of region in transition. > Then the balancer use the computed value to throttling. > Examples for understanding. > A cluster has 100 regionservers, each regionserver has 200 regions and the > average time of region in transition is 1 seconds, we config the max > balancing time is 10 * 60 seconds. > Case 1. One regionserver crash, the cluster at most need balance 200 regions. > Then 200 / (10 * 60s / 1s) < 1, it means the max number of regions in > transition is 1 when balancing. Then the balancer can move region one by one > and the cluster will have high availability when balancing. > Case 2. Add other 100 regionservers, the cluster at most need balance 1 > regions. Then 1 / (10 * 60s / 1s) = 16.7, it means the max number of > regions in transition is 17 when balancing. Then the cluster can reach a > balance state within the max balancing time. > Any suggestions are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17181) Let HBase thrift2 support TThreadedSelectorServer
[
https://issues.apache.org/jira/browse/HBASE-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704045#comment-15704045
]
Hadoop QA commented on HBASE-17181:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HBASE-17181 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12840790/HBASE-17181-V3.patch |
| JIRA Issue | HBASE-17181 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4667/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Let HBase thrift2 support TThreadedSelectorServer
> -
>
> Key: HBASE-17181
> URL: https://issues.apache.org/jira/browse/HBASE-17181
> Project: HBase
> Issue Type: New Feature
> Components: Thrift
>Affects Versions: 1.2.3
>Reporter: Jian Yi
>Priority: Minor
> Labels: features
> Fix For: 1.2.3
>
> Attachments: HBASE-17181-V1.patch, HBASE-17181-V2.patch,
> HBASE-17181-V3.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Add TThreadedSelectorServer for HBase Thrift2
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-17181) Let HBase thrift2 support TThreadedSelectorServer
[ https://issues.apache.org/jira/browse/HBASE-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Yi updated HBASE-17181: Attachment: HBASE-17181-V3.patch use curly braces around the method call > Let HBase thrift2 support TThreadedSelectorServer > - > > Key: HBASE-17181 > URL: https://issues.apache.org/jira/browse/HBASE-17181 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 1.2.3 >Reporter: Jian Yi >Priority: Minor > Labels: features > Fix For: 1.2.3 > > Attachments: HBASE-17181-V1.patch, HBASE-17181-V2.patch, > HBASE-17181-V3.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Add TThreadedSelectorServer for HBase Thrift2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2
[ https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704021#comment-15704021 ] Phil Yang commented on HBASE-17182: --- ChoreService can not record the last access time of scanners. Guava's cache is simpler in this scene. > Memory leak from openScanner of HBase thrift2 > - > > Key: HBASE-17182 > URL: https://issues.apache.org/jira/browse/HBASE-17182 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: Jian Yi > > Client call openScanner, but client (coredump or others) not closeScanner, > the scanner will not be removed from scannerMap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17167) Pass mvcc to client when scan
[ https://issues.apache.org/jira/browse/HBASE-17167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17167: -- Attachment: HBASE-17167-v4.patch Always set mvccReadPoint to -1 when reset. > Pass mvcc to client when scan > - > > Key: HBASE-17167 > URL: https://issues.apache.org/jira/browse/HBASE-17167 > Project: HBase > Issue Type: Sub-task > Components: Client, scan >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17167-v1.patch, HBASE-17167-v2.patch, > HBASE-17167-v3.patch, HBASE-17167-v4.patch, HBASE-17167.patch > > > For the current implementation, if we use batch or allowPartial when scan, > then the row level atomic can not be guaranteed if we need to restart a scan > in the middle of a record due to region move or something else. > We can return the mvcc used to open scanner to client and client could use > this mvcc to restart a scan to get row level atomic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17048) Introduce a helper class to calcuate suitable ByteBuf size when allocating send buffer in FanOutOneBlockAsyncDFSOutputHelper
[ https://issues.apache.org/jira/browse/HBASE-17048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704014#comment-15704014 ] Duo Zhang commented on HBASE-17048: --- +1. > Introduce a helper class to calcuate suitable ByteBuf size when allocating > send buffer in FanOutOneBlockAsyncDFSOutputHelper > > > Key: HBASE-17048 > URL: https://issues.apache.org/jira/browse/HBASE-17048 > Project: HBase > Issue Type: Sub-task > Components: wal >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-17012_3.patch, HBASE-17048.patch, > HBASE-17048_1.patch, HBASE-17048_3.patch > > > As [~ram_krish] mentioned in HBASE-17021 > https://issues.apache.org/jira/browse/HBASE-17021?focusedCommentId=15646938=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15646938 > The default ByteBuf size is 256B which is too small. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17086) Add comments to explain why Cell#getTagsLength() returns an int, rather than a short
[ https://issues.apache.org/jira/browse/HBASE-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704005#comment-15704005 ] Xiang Li commented on HBASE-17086: -- [~anoop.hbase], would you please check if the patch also makes sense to you? > Add comments to explain why Cell#getTagsLength() returns an int, rather than > a short > > > Key: HBASE-17086 > URL: https://issues.apache.org/jira/browse/HBASE-17086 > Project: HBase > Issue Type: Improvement > Components: Interface >Affects Versions: 2.0.0 >Reporter: Xiang Li >Assignee: Xiang Li >Priority: Minor > Attachments: HBASE-17086.master.000.patch > > > In the Cell interface, getTagsLength() returns a int > But in the KeyValue implementation, tags length is of 2 bytes. Also in > ExtendedCell, when explaining the KeyValue format, tags length is stated to > be 2 bytes > Any plan to update Cell interface to make getTagsLength() returns a short ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17187) DoNotRetryExceptions from coprocessors should bubble up to the application
[
https://issues.apache.org/jira/browse/HBASE-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703984#comment-15703984
]
Enis Soztutar commented on HBASE-17187:
---
Thanks Josh for taking a look.
bq. From the context of the Apache Phoenix test failure which caught this:
making the RPC 30 more times (w/ default configs) is not-ideal.
We are using the default num of retries only in the case where the exception is
thrown is NOT a DNRIOE. Otherwise, we do not retry. The Phoenix test failure is
throwing DNRIOE I think.
bq. Long-term, makes me think defining exceptions that will come out of
RSpcServices#scan(...) is a good idea for coprocessors to hook into for more
"application-level" exceptions.
Agreed.
> DoNotRetryExceptions from coprocessors should bubble up to the application
> --
>
> Key: HBASE-17187
> URL: https://issues.apache.org/jira/browse/HBASE-17187
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-17187_v1.patch
>
>
> In HBASE-16604, we fixed a case where scanner retries was causing the scan to
> miss some data in case the scanner is left with a dirty state (like a
> half-seeked KVHeap).
> The patch introduced a minor compatibility issue, because now if a
> coprocessor throws DNRIOE, we still retry the ClientScanner indefinitely.
> The test {{ServerExceptionIT}} in Phoenix is failing because of this with
> HBASE-16604.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17189) TestMasterObserver#wasModifyTableActionCalled uses wrong variables
[
https://issues.apache.org/jira/browse/HBASE-17189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703976#comment-15703976
]
Hadoop QA commented on HBASE-17189:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
33s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
23m 28s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 90m 41s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 125m 22s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12840760/HBASE-17189.v1-master.patch
|
| JIRA Issue | HBASE-17189 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 4fb44ede719a 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 3d5e686 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4659/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4659/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> TestMasterObserver#wasModifyTableActionCalled uses wrong variables
> --
>
> Key: HBASE-17189
> URL: https://issues.apache.org/jira/browse/HBASE-17189
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Stephen Yuan Jiang
>Assignee: Stephen Yuan Jiang
>Priority: Minor
> Attachments:
[jira] [Updated] (HBASE-17110) Add an "Overall Strategy" option(balanced both on table level and server level) to SimpleLoadBalancer
[
https://issues.apache.org/jira/browse/HBASE-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charlie Qiangeng Xu updated HBASE-17110:
Attachment: HBASE-17110-V8.patch
> Add an "Overall Strategy" option(balanced both on table level and server
> level) to SimpleLoadBalancer
> -
>
> Key: HBASE-17110
> URL: https://issues.apache.org/jira/browse/HBASE-17110
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
>Affects Versions: 2.0.0, 1.2.4
>Reporter: Charlie Qiangeng Xu
>Assignee: Charlie Qiangeng Xu
> Attachments: HBASE-17110-V2.patch, HBASE-17110-V3.patch,
> HBASE-17110-V4.patch, HBASE-17110-V5.patch, HBASE-17110-V6.patch,
> HBASE-17110-V7.patch, HBASE-17110-V8.patch, HBASE-17110.patch
>
>
> This jira is about an enhancement of simpleLoadBalancer. Here we introduce a
> new strategy: "bytableOverall" which could be controlled by adding:
> {noformat}
>
> hbase.master.loadbalance.bytableOverall
> true
>
> {noformat}
> We have been using the strategy on our largest cluster for several months.
> it's proven to be very helpful and stable, especially, the result is quite
> visible to the users.
> Here is the reason why it's helpful:
> When operating large scale clusters(our case), some companies still prefer to
> use {{SimpleLoadBalancer}} due to its simplicity, quick balance plan
> generation, etc. Current SimpleLoadBalancer has two modes:
> 1. byTable, which only guarantees that the regions of one table could be
> uniformly distributed.
> 2. byCluster, which ignores the distribution within tables and balance the
> regions all together.
> If the pressures on different tables are different, the first byTable option
> is the preferable one in most case. Yet, this choice sacrifice the cluster
> level balance and would cause some servers to have significantly higher load,
> e.g. 242 regions on server A but 417 regions on server B.(real world stats)
> Consider this case, a cluster has 3 tables and 4 servers:
> {noformat}
> server A has 3 regions: table1:1, table2:1, table3:1
> server B has 3 regions: table1:2, table2:2, table3:2
> server C has 3 regions: table1:3, table2:3, table3:3
> server D has 0 regions.
> {noformat}
> From the byTable strategy's perspective, the cluster has already been
> perfectly balanced on table level. But a perfect status should be like:
> {noformat}
> server A has 2 regions: table2:1, table3:1
> server B has 2 regions: table1:2, table3:2
> server C has 3 regions: table1:3, table2:3, table3:3
> server D has 2 regions: table1:1, table2:2
> {noformat}
> We can see the server loads change from 3,3,3,0 to 2,2,3,2, while the table1,
> table2 and table3 still keep balanced.
> And this is what the new mode "byTableOverall" can achieve.
> Two UTs have been added as well and the last one demonstrates the advantage
> of the new strategy.
> Also, a onConfigurationChange method has been implemented to hot control the
> "slop" variable.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-17187) DoNotRetryExceptions from coprocessors should bubble up to the application
[
https://issues.apache.org/jira/browse/HBASE-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-17187:
--
Status: Patch Available (was: Open)
> DoNotRetryExceptions from coprocessors should bubble up to the application
> --
>
> Key: HBASE-17187
> URL: https://issues.apache.org/jira/browse/HBASE-17187
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-17187_v1.patch
>
>
> In HBASE-16604, we fixed a case where scanner retries was causing the scan to
> miss some data in case the scanner is left with a dirty state (like a
> half-seeked KVHeap).
> The patch introduced a minor compatibility issue, because now if a
> coprocessor throws DNRIOE, we still retry the ClientScanner indefinitely.
> The test {{ServerExceptionIT}} in Phoenix is failing because of this with
> HBASE-16604.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17187) DoNotRetryExceptions from coprocessors should bubble up to the application
[
https://issues.apache.org/jira/browse/HBASE-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703948#comment-15703948
]
Josh Elser commented on HBASE-17187:
[~enis], your patch looks like it would work to me. Some thoughts in reading
over it:
* Great comments in-line. These do a fantastic job explaining why it is the way
it is.
* From the context of the Apache Phoenix test failure which caught this: making
the RPC 30 more times (w/ default configs) is not-ideal.
* Long-term, makes me think defining exceptions that will come out of
{{RSpcServices#scan(...)}} is a good idea for coprocessors to hook into for
more "application-level" exceptions. e.g. something like
{{UnknownScannerException}}, {{ScannerResetException}}, and a
{{CoprocessorApplicationException}} which we would know to propagate and
clients could know to not retry. Essentially, helps us remove some of the
ambiguity around DNRE that you explained.
> DoNotRetryExceptions from coprocessors should bubble up to the application
> --
>
> Key: HBASE-17187
> URL: https://issues.apache.org/jira/browse/HBASE-17187
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-17187_v1.patch
>
>
> In HBASE-16604, we fixed a case where scanner retries was causing the scan to
> miss some data in case the scanner is left with a dirty state (like a
> half-seeked KVHeap).
> The patch introduced a minor compatibility issue, because now if a
> coprocessor throws DNRIOE, we still retry the ClientScanner indefinitely.
> The test {{ServerExceptionIT}} in Phoenix is failing because of this with
> HBASE-16604.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17181) Let HBase thrift2 support TThreadedSelectorServer
[
https://issues.apache.org/jira/browse/HBASE-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703934#comment-15703934
]
Hadoop QA commented on HBASE-17181:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HBASE-17181 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12840780/HBASE-17181-V2.patch |
| JIRA Issue | HBASE-17181 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4663/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Let HBase thrift2 support TThreadedSelectorServer
> -
>
> Key: HBASE-17181
> URL: https://issues.apache.org/jira/browse/HBASE-17181
> Project: HBase
> Issue Type: New Feature
> Components: Thrift
>Affects Versions: 1.2.3
>Reporter: Jian Yi
>Priority: Minor
> Labels: features
> Fix For: 1.2.3
>
> Attachments: HBASE-17181-V1.patch, HBASE-17181-V2.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Add TThreadedSelectorServer for HBase Thrift2
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-17181) Let HBase thrift2 support TThreadedSelectorServer
[ https://issues.apache.org/jira/browse/HBASE-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Yi updated HBASE-17181: Attachment: HBASE-17181-V2.patch patch for master > Let HBase thrift2 support TThreadedSelectorServer > - > > Key: HBASE-17181 > URL: https://issues.apache.org/jira/browse/HBASE-17181 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 1.2.3 >Reporter: Jian Yi >Priority: Minor > Labels: features > Fix For: 1.2.3 > > Attachments: HBASE-17181-V1.patch, HBASE-17181-V2.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Add TThreadedSelectorServer for HBase Thrift2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17185) Purge the seek of the next block reading HFileBlocks
[
https://issues.apache.org/jira/browse/HBASE-17185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703920#comment-15703920
]
Hadoop QA commented on HBASE-17185:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 5s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 24s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 28m 19s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 74m 33s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.io.hfile.TestHFileBlockPositionalRead |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12840765/HBASE-17185.master.001.patch
|
| JIRA Issue | HBASE-17185 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 826342fa0780 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 3d5e686 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/4660/artifact/patchprocess/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/4660/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/4660/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4660/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4660/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message
[jira] [Commented] (HBASE-17178) Add region balance throttling
[ https://issues.apache.org/jira/browse/HBASE-17178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703912#comment-15703912 ] Guanghao Zhang commented on HBASE-17178: Good idea. When RIT duration is longer than we expected, then there are more concurrent RIT regions. This will decrease the cluster availability... There is a trade-off between availability and how long to reach balance state. > Add region balance throttling > - > > Key: HBASE-17178 > URL: https://issues.apache.org/jira/browse/HBASE-17178 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17178-v1.patch > > > Our online cluster serves dozens of tables and different tables serve for > different services. If the balancer moves too many regions in the same time, > it will decrease the availability for some table or some services. So we add > region balance throttling on our online serve cluster. > We introduce a new config hbase.balancer.max.balancing.regions, which means > the max number of regions in transition when balancing. > If we config this to 1 and a table have 100 regions, then the table will have > 99 regions available at any time. It helps a lot for our use case and it has > been running a long time > our production cluster. > But for some use case, we need the balancer run faster. If a cluster has 100 > regionservers, then it add 50 new regionservers for peak requests. Then it > need balancer run as soon as > possible and let the cluster reach a balance state soon. Our idea is compute > max number of regions in transition by the max balancing time and the average > time of region in transition. > Then the balancer use the computed value to throttling. > Examples for understanding. > A cluster has 100 regionservers, each regionserver has 200 regions and the > average time of region in transition is 1 seconds, we config the max > balancing time is 10 * 60 seconds. > Case 1. One regionserver crash, the cluster at most need balance 200 regions. > Then 200 / (10 * 60s / 1s) < 1, it means the max number of regions in > transition is 1 when balancing. Then the balancer can move region one by one > and the cluster will have high availability when balancing. > Case 2. Add other 100 regionservers, the cluster at most need balance 1 > regions. Then 1 / (10 * 60s / 1s) = 16.7, it means the max number of > regions in transition is 17 when balancing. Then the cluster can reach a > balance state within the max balancing time. > Any suggestions are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17072) CPU usage starts to climb up to 90-100% when using G1GC; purge ThreadLocal usage
[
https://issues.apache.org/jira/browse/HBASE-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703902#comment-15703902
]
Hudson commented on HBASE-17072:
SUCCESS: Integrated in Jenkins build HBase-1.4 #548 (See
[https://builds.apache.org/job/HBase-1.4/548/])
HBASE-17072 CPU usage starts to climb up to 90-100% when using G1GC (stack: rev
987205caf9071b35239e803138de8a87913e6cae)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
> CPU usage starts to climb up to 90-100% when using G1GC; purge ThreadLocal
> usage
>
>
> Key: HBASE-17072
> URL: https://issues.apache.org/jira/browse/HBASE-17072
> Project: HBase
> Issue Type: Bug
> Components: Performance, regionserver
>Affects Versions: 1.0.0, 2.0.0, 1.2.0
>Reporter: Eiichi Sato
>Assignee: Eiichi Sato
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17072.branch-1.001.patch,
> HBASE-17072.master.001.patch, HBASE-17072.master.002.patch,
> HBASE-17072.master.003.patch, HBASE-17072.master.004.patch,
> HBASE-17072.master.005.patch, HBASE-17072.master.005.patch,
> disable-block-header-cache.patch, mat-threadlocals.png, mat-threads.png,
> metrics.png, slave1.svg, slave2.svg, slave3.svg, slave4.svg
>
>
> h5. Problem
> CPU usage of a region server in our CDH 5.4.5 cluster, at some point, starts
> to gradually get higher up to nearly 90-100% when using G1GC. We've also run
> into this problem on CDH 5.7.3 and CDH 5.8.2.
> In our production cluster, it normally takes a few weeks for this to happen
> after restarting a RS. We reproduced this on our test cluster and attached
> the results. Please note that, to make it easy to reproduce, we did some
> "anti-tuning" on a table when running tests.
> In metrics.png, soon after we started running some workloads against a test
> cluster (CDH 5.8.2) at about 7 p.m. CPU usage of the two RSs started to rise.
> Flame Graphs (slave1.svg to slave4.svg) are generated from jstack dumps of
> each RS process around 10:30 a.m. the next day.
> After investigating heapdumps from another occurrence on a test cluster
> running CDH 5.7.3, we found that the ThreadLocalMap contain a lot of
> contiguous entries of {{HFileBlock$PrefetchedHeader}} probably due to primary
> clustering. This caused more loops in
> {{ThreadLocalMap#expungeStaleEntries()}}, consuming a certain amount of CPU
> time. What is worse is that the method is called from RPC metrics code,
> which means even a small amount of per-RPC time soon adds up to a huge amount
> of CPU time.
> This is very similar to the issue in HBASE-16616, but we have many
> {{HFileBlock$PrefetchedHeader}} not only {{Counter$IndexHolder}} instances.
> Here are some OQL counts from Eclipse Memory Analyzer (MAT). This shows a
> number of ThreadLocal instances in the ThreadLocalMap of a single handler
> thread.
> {code}
> SELECT *
> FROM OBJECTS (SELECT AS RETAINED SET OBJECTS value
> FROM OBJECTS 0x4ee380430) obj
> WHERE obj.@clazz.@name =
> "org.apache.hadoop.hbase.io.hfile.HFileBlock$PrefetchedHeader"
> #=> 10980 instances
> {code}
> {code}
> SELECT *
> FROM OBJECTS (SELECT AS RETAINED SET OBJECTS value
> FROM OBJECTS 0x4ee380430) obj
> WHERE obj.@clazz.@name = "org.apache.hadoop.hbase.util.Counter$IndexHolder"
> #=> 2052 instances
> {code}
> Although as described in HBASE-16616 this somewhat seems to be an issue in
> G1GC side regarding weakly-reachable objects, we should keep ThreadLocal
> usage minimal and avoid creating an indefinite number (in this case, a number
> of HFiles) of ThreadLocal instances.
> HBASE-16146 removes ThreadLocals from the RPC metrics code. That may solve
> the issue (I just saw the patch, never tested it at all), but the
> {{HFileBlock$PrefetchedHeader}} are still there in the ThreadLocalMap, which
> may cause issues in the future again.
> h5. Our Solution
> We simply removed the whole {{HFileBlock$PrefetchedHeader}} caching and
> fortunately we didn't notice any performance degradation for our production
> workloads.
> Because the PrefetchedHeader caching uses ThreadLocal and because RPCs are
> handled randomly in any of the handlers, small Get or small Scan RPCs do not
> benefit from the caching (See HBASE-10676 and HBASE-11402 for the details).
> Probably, we need to see how well reads are saved by the caching for large
> Scan or Get RPCs and especially for compactions if we really remove the
> caching. It's probably better if we can remove ThreadLocals without breaking
> the current caching behavior.
> FWIW, I'm attaching the patch we applied. It's for CDH
[jira] [Updated] (HBASE-17187) DoNotRetryExceptions from coprocessors should bubble up to the application
[
https://issues.apache.org/jira/browse/HBASE-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-17187:
--
Attachment: hbase-17187_v1.patch
v1 patch which rethrows the DNRIOE from the server side and introduces retry
limits in the client scanner retries.
[~elserj] do you mind taking a look (from offline discussions).
> DoNotRetryExceptions from coprocessors should bubble up to the application
> --
>
> Key: HBASE-17187
> URL: https://issues.apache.org/jira/browse/HBASE-17187
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-17187_v1.patch
>
>
> In HBASE-16604, we fixed a case where scanner retries was causing the scan to
> miss some data in case the scanner is left with a dirty state (like a
> half-seeked KVHeap).
> The patch introduced a minor compatibility issue, because now if a
> coprocessor throws DNRIOE, we still retry the ClientScanner indefinitely.
> The test {{ServerExceptionIT}} in Phoenix is failing because of this with
> HBASE-16604.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17172) Optimize major mob compaction with _del files
[ https://issues.apache.org/jira/browse/HBASE-17172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703892#comment-15703892 ] Jingcheng Du commented on HBASE-17172: -- Thanks [~huaxiang] and [~anoopsamjohn]. bq. Means the deleted MOB cells will be there in MOB files unless they are TTL expired? Right. But according to Huaxiang's comments, this is not doable. So it seems merging files by regions is the only way now. > Optimize major mob compaction with _del files > - > > Key: HBASE-17172 > URL: https://issues.apache.org/jira/browse/HBASE-17172 > Project: HBase > Issue Type: Improvement > Components: mob >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > > Today, when there is a _del file in mobdir, with major mob compaction, every > mob file will be recompacted, this causes lots of IO and slow down major mob > compaction (may take months to finish). This needs to be improved. A few > ideas are: > 1) Do not compact all _del files into one, instead, compact them based on > groups with startKey as the key. Then use firstKey/startKey to make each mob > file to see if the _del file needs to be included for this partition. > 2). Based on the timerange of the _del file, compaction for files after that > timerange does not need to include the _del file as these are newer files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17186) MasterProcedureTestingUtility#testRecoveryAndDoubleExecution displays stale procedure state info
[
https://issues.apache.org/jira/browse/HBASE-17186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703884#comment-15703884
]
Hadoop QA commented on HBASE-17186:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 2s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 14s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 100m 2s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 147m 32s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12840751/HBASE-17186.v2-master.patch
|
| JIRA Issue | HBASE-17186 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux fa9a2b89d28f 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 3d5e686 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4656/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4656/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> MasterProcedureTestingUtility#testRecoveryAndDoubleExecution displays stale
> procedure state info
>
>
> Key: HBASE-17186
> URL: https://issues.apache.org/jira/browse/HBASE-17186
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, test
>Reporter: Stephen Yuan Jiang
>Assignee: Stephen Yuan
[jira] [Updated] (HBASE-16561) Add metrics about read/write/scan queue length and active read/write/scan handler count
[ https://issues.apache.org/jira/browse/HBASE-16561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-16561: --- Attachment: HBASE-16561-branch-1.patch Attach patch for branch-1. > Add metrics about read/write/scan queue length and active read/write/scan > handler count > --- > > Key: HBASE-16561 > URL: https://issues.apache.org/jira/browse/HBASE-16561 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, metrics >Affects Versions: 2.0.0, 1.4.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16561-branch-1.patch, HBASE-16561-v1.patch, > HBASE-16561.patch > > > Now there are only metrics about total queue length and active rpc handler > count. But in the RWQueueRpcExecutor, there are different queues and handlers > for read/write/scan request. I thought it is necessary to add more metrics > for RWQueueRpcExecutor. When use it in production cluster, we can adjust the > config of queues and handlers according to the metrics. > Review url: https://reviews.apache.org/r/54072/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-17057) Minor compactions should also drop page cache behind reads
[ https://issues.apache.org/jira/browse/HBASE-17057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703800#comment-15703800 ] Ashu Pachauri edited comment on HBASE-17057 at 11/29/16 1:59 AM: - I had a first look at the code, things never are straightforward in the land of compactions. 1. Whether we can drop cache behind reads/writes for compactions is tied to whether we should throttle compactions. Not sure what dropping page cache has to do with throttling compactions. [~eclark] , can you shed a little light on why this is the case? (Since you added this piece of code in HBASE-14098 ) 2. It seems that we just ignore drop cache hint for all compactions on master branch. I'll dig more and try to find why the behavior was modified on master. was (Author: ashu210890): I had a first look at the code, things never are straightforward in the land of compactions. 1. Whether we can drop cache behind reads/writes for compaction is tried to whether we should throttle compactions. Not sure what dropping page cache has to do with throttling compactions. [~eclark] , can you shed a little light on why this is the case? (Since you added this piece of code in HBASE-14098 ) 2. It seems that we just ignore drop cache hint for all compactions on master branch. I'll dig more and try to find why the behavior was modified on master. > Minor compactions should also drop page cache behind reads > -- > > Key: HBASE-17057 > URL: https://issues.apache.org/jira/browse/HBASE-17057 > Project: HBase > Issue Type: Improvement > Components: Compaction >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > > Long compactions currently drop cache behind reads so that they don't pollute > the page cache but short compactions don't do that. The bulk of the data is > actually read during minor compactions instead of major compactions, and > thrashes the page cache since it's mostly not needed. > We should drop page cache behind reads for minor compactions too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16755) Honor flush policy under global memstore pressure
[ https://issues.apache.org/jira/browse/HBASE-16755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashu Pachauri updated HBASE-16755: -- Status: Patch Available (was: Open) > Honor flush policy under global memstore pressure > - > > Key: HBASE-16755 > URL: https://issues.apache.org/jira/browse/HBASE-16755 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > Attachments: HBASE-16755.v0.patch > > > When global memstore reaches the low water mark, we pick the best flushable > region and flush all column families for it. This is a suboptimal approach in > the sense that it leads to an unnecessarily high file creation rate and IO > amplification due to compactions. We should still try to honor the underlying > FlushPolicy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17181) Let HBase thrift2 support TThreadedSelectorServer
[ https://issues.apache.org/jira/browse/HBASE-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703859#comment-15703859 ] Duo Zhang commented on HBASE-17181: --- Can you compile master without the patch? The problem is missing dependency I think. Do you have a settings.xml under ~/.m2? You can try renaming it to another name and try compiling again. > Let HBase thrift2 support TThreadedSelectorServer > - > > Key: HBASE-17181 > URL: https://issues.apache.org/jira/browse/HBASE-17181 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 1.2.3 >Reporter: Jian Yi >Priority: Minor > Labels: features > Fix For: 1.2.3 > > Attachments: HBASE-17181-V1.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Add TThreadedSelectorServer for HBase Thrift2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16755) Honor flush policy under global memstore pressure
[ https://issues.apache.org/jira/browse/HBASE-16755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashu Pachauri updated HBASE-16755: -- Attachment: HBASE-16755.v0.patch > Honor flush policy under global memstore pressure > - > > Key: HBASE-16755 > URL: https://issues.apache.org/jira/browse/HBASE-16755 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > Attachments: HBASE-16755.v0.patch > > > When global memstore reaches the low water mark, we pick the best flushable > region and flush all column families for it. This is a suboptimal approach in > the sense that it leads to an unnecessarily high file creation rate and IO > amplification due to compactions. We should still try to honor the underlying > FlushPolicy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17178) Add region balance throttling
[ https://issues.apache.org/jira/browse/HBASE-17178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703856#comment-15703856 ] Dave Latham commented on HBASE-17178: - The goal of both, from what I can tell is to limit the unavailability of regions during balancing. I'm happy to see the issue picked up again, and hope an implementation makes it in this time. > Add region balance throttling > - > > Key: HBASE-17178 > URL: https://issues.apache.org/jira/browse/HBASE-17178 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17178-v1.patch > > > Our online cluster serves dozens of tables and different tables serve for > different services. If the balancer moves too many regions in the same time, > it will decrease the availability for some table or some services. So we add > region balance throttling on our online serve cluster. > We introduce a new config hbase.balancer.max.balancing.regions, which means > the max number of regions in transition when balancing. > If we config this to 1 and a table have 100 regions, then the table will have > 99 regions available at any time. It helps a lot for our use case and it has > been running a long time > our production cluster. > But for some use case, we need the balancer run faster. If a cluster has 100 > regionservers, then it add 50 new regionservers for peak requests. Then it > need balancer run as soon as > possible and let the cluster reach a balance state soon. Our idea is compute > max number of regions in transition by the max balancing time and the average > time of region in transition. > Then the balancer use the computed value to throttling. > Examples for understanding. > A cluster has 100 regionservers, each regionserver has 200 regions and the > average time of region in transition is 1 seconds, we config the max > balancing time is 10 * 60 seconds. > Case 1. One regionserver crash, the cluster at most need balance 200 regions. > Then 200 / (10 * 60s / 1s) < 1, it means the max number of regions in > transition is 1 when balancing. Then the balancer can move region one by one > and the cluster will have high availability when balancing. > Case 2. Add other 100 regionservers, the cluster at most need balance 1 > regions. Then 1 / (10 * 60s / 1s) = 16.7, it means the max number of > regions in transition is 17 when balancing. Then the cluster can reach a > balance state within the max balancing time. > Any suggestions are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16755) Honor flush policy under global memstore pressure
[ https://issues.apache.org/jira/browse/HBASE-16755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashu Pachauri updated HBASE-16755: -- Summary: Honor flush policy under global memstore pressure (was: Don't flush everything when under global memstore pressure) > Honor flush policy under global memstore pressure > - > > Key: HBASE-16755 > URL: https://issues.apache.org/jira/browse/HBASE-16755 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > > When global memstore reaches the low water mark, we pick the best flushable > region and flush all column families for it. This is a suboptimal approach in > the sense that it leads to an unnecessarily high file creation rate and IO > amplification due to compactions. We should still try to honor the underlying > FlushPolicy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17180) Let HBase thrift2 support TThreadedSelectorServer
[ https://issues.apache.org/jira/browse/HBASE-17180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703847#comment-15703847 ] Jian Yi commented on HBASE-17180: - sorry, first create issue > Let HBase thrift2 support TThreadedSelectorServer > - > > Key: HBASE-17180 > URL: https://issues.apache.org/jira/browse/HBASE-17180 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 1.2.3 >Reporter: Jian Yi >Priority: Minor > Labels: features > Fix For: 1.2.3 > > Original Estimate: 2h > Remaining Estimate: 2h > > Add TThreadedSelectorServer for HBase Thrift2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17181) Let HBase thrift2 support TThreadedSelectorServer
[ https://issues.apache.org/jira/browse/HBASE-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703845#comment-15703845 ] Jian Yi commented on HBASE-17181: - without similar unit test in TestThriftHBaseServiceHandler.java about Server, I compile and test it on 1.2.3 version online. When I compile it on master, meeting many error: I compiled it on 1.2.3, I meet compile error on master: ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project hbase-thrift: Execution default-compile of goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile failed: A required class was missing while executing org.apache.maven.plugins:maven-compiler-plugin:3.2:compile: org/codehaus/plexus/compiler/util/scan/InclusionScanException [ERROR] - [ERROR] realm =plugin>org.apache.maven.plugins:maven-compiler-plugin:3.2 [ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy [ERROR] urls[0] = file:/C:/Users/jianyi/.m2/repository/org/apache/maven/plugins/maven-compiler-plugin/3.2/maven-compiler-plugin-3.2.jar [ERROR] urls[1] = file:/C:/Users/jianyi/.m2/repository/org/codehaus/plexus/plexus-utils/1.5.1/plexus-utils-1.5.1.jar [ERROR] urls[2] = file:/C:/Users/jianyi/.m2/repository/org/apache/maven/shared/maven-shared-utils/0.1/maven-shared-utils-0.1.jar [ERROR] urls[3] = file:/C:/Users/jianyi/.m2/repository/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar [ERROR] urls[4] = file:/C:/Users/jianyi/.m2/repository/org/codehaus/plexus/plexus-compiler-api/2.4/plexus-compiler-api-2.4.jar [ERROR] urls[5] = file:/C:/Users/jianyi/.m2/repository/org/codehaus/plexus/plexus-compiler-manager/2.4/plexus-compiler-manager-2.4.jar [ERROR] urls[6] = file:/C:/Users/jianyi/.m2/repository/org/codehaus/plexus/plexus-compiler-javac/2.4/plexus-compiler-javac-2.4.jar [ERROR] urls[7] = file:/C:/Users/jianyi/.m2/repository/org/apache/xbean/xbean-reflect/3.4/xbean-reflect-3.4.jar [ERROR] urls[8] = file:/C:/Users/jianyi/.m2/repository/log4j/log4j/1.2.12/log4j-1.2.12.jar [ERROR] urls[9] = file:/C:/Users/jianyi/.m2/repository/commons-logging/commons-logging-api/1.1/commons-logging-api-1.1.jar [ERROR] urls[10] = file:/C:/Users/jianyi/.m2/repository/com/google/collections/google-collections/1.0/google-collections-1.0.jar [ERROR] urls[11] = file:/C:/Users/jianyi/.m2/repository/junit/junit/3.8.2/junit-3.8.2.jar [ERROR] Number of foreign imports: 1 [ERROR] import: Entry[import from realm ClassRealm[project>org.apache.hbase:hbase-thrift:2.0.0-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]] [ERROR] [ERROR] -: org.codehaus.plexus.compiler.util.scan.InclusionScanException [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginContainerException > Let HBase thrift2 support TThreadedSelectorServer > - > > Key: HBASE-17181 > URL: https://issues.apache.org/jira/browse/HBASE-17181 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 1.2.3 >Reporter: Jian Yi >Priority: Minor > Labels: features > Fix For: 1.2.3 > > Attachments: HBASE-17181-V1.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Add TThreadedSelectorServer for HBase Thrift2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16561) Add metrics about read/write/scan queue length and active read/write/scan handler count
[ https://issues.apache.org/jira/browse/HBASE-16561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-16561: -- Affects Version/s: 1.4.0 2.0.0 Fix Version/s: 1.4.0 2.0.0 > Add metrics about read/write/scan queue length and active read/write/scan > handler count > --- > > Key: HBASE-16561 > URL: https://issues.apache.org/jira/browse/HBASE-16561 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, metrics >Affects Versions: 2.0.0, 1.4.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16561-v1.patch, HBASE-16561.patch > > > Now there are only metrics about total queue length and active rpc handler > count. But in the RWQueueRpcExecutor, there are different queues and handlers > for read/write/scan request. I thought it is necessary to add more metrics > for RWQueueRpcExecutor. When use it in production cluster, we can adjust the > config of queues and handlers according to the metrics. > Review url: https://reviews.apache.org/r/54072/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16561) Add metrics about read/write/scan queue length and active read/write/scan handler count
[ https://issues.apache.org/jira/browse/HBASE-16561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703840#comment-15703840 ] Duo Zhang commented on HBASE-16561: --- Pushed to master, but the patch can not be applied to branch-1 cleanly. Could you please prepare a patch for branch-1 [~zghaobac]? Thanks. > Add metrics about read/write/scan queue length and active read/write/scan > handler count > --- > > Key: HBASE-16561 > URL: https://issues.apache.org/jira/browse/HBASE-16561 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, metrics >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-16561-v1.patch, HBASE-16561.patch > > > Now there are only metrics about total queue length and active rpc handler > count. But in the RWQueueRpcExecutor, there are different queues and handlers > for read/write/scan request. I thought it is necessary to add more metrics > for RWQueueRpcExecutor. When use it in production cluster, we can adjust the > config of queues and handlers according to the metrics. > Review url: https://reviews.apache.org/r/54072/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17179) the chunkpool are not reclamed correctly
[
https://issues.apache.org/jira/browse/HBASE-17179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703832#comment-15703832
]
chenxu commented on HBASE-17179:
the global memstore never crosses 3 GB, so the _reclaimedChunks_ should not
overflow,
and the heap size should always biger than chunkpool's initial size i think,
but it's not.
some chunks are collected as garbage, not put back to the pool.
> the chunkpool are not reclamed correctly
>
>
> Key: HBASE-17179
> URL: https://issues.apache.org/jira/browse/HBASE-17179
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.0.1.1, 1.1.2
>Reporter: chenxu
>
> in our cluster, the RS's config is as below
> {quote}
> heap size(25G)
> hbase.regionserver.global.memstore.size(0.5)
> hbase.hregion.memstore.chunkpool.maxsize(0.8)
> hbase.hregion.memstore.chunkpool.initialsize(1.0)
> {quote}
> so the chunkpool's size is 25*0.5*0.8 = 10GB
> in our monitor system, the _memStoreSize_ never up to 3G，so the _reuseRatio_
> of the chunkpool should always 100%, but it's not, see the logs below:
> {quote}
> 2016-11-23 11:47:34,302 DEBUG
> [StoreOpener-33a77b971aff15e5e6be3cc614870e43-1-MemStoreChunkPool Statistics]
> regionserver.MemStoreChunkPool: Stats: current pool size=5009,created chunk
> count=0,reused chunk count=81,reuseRatio=100.00%
> 2016-11-28 11:47:34,302 DEBUG
> [StoreOpener-33a77b971aff15e5e6be3cc614870e43-1-MemStoreChunkPool Statistics]
> regionserver.MemStoreChunkPool: Stats: current pool size=3,created chunk
> count=1325,reused chunk count=67209,reuseRatio=98.07%
> {quote}
> 5 days later, the _reuseRatio_ is below 100%
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-17181) Let HBase thrift2 support TThreadedSelectorServer
[ https://issues.apache.org/jira/browse/HBASE-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703823#comment-15703823 ] Jian Yi edited comment on HBASE-17181 at 11/29/16 1:42 AM: --- I compiled it on 1.2.3, I meet compile error on master: ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project hbase-thrift: Execution default-compile of goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile failed: A required class was missing while executing org.apache.maven.plugins:maven-compiler-plugin:3.2:compile: org/codehaus/plexus/compiler/util/scan/InclusionScanException [ERROR] - [ERROR] realm =plugin>org.apache.maven.plugins:maven-compiler-plugin:3.2 [ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy [ERROR] urls[0] = file:/C:/Users/jianyi/.m2/repository/org/apache/maven/plugins/maven-compiler-plugin/3.2/maven-compiler-plugin-3.2.jar [ERROR] urls[1] = file:/C:/Users/jianyi/.m2/repository/org/codehaus/plexus/plexus-utils/1.5.1/plexus-utils-1.5.1.jar [ERROR] urls[2] = file:/C:/Users/jianyi/.m2/repository/org/apache/maven/shared/maven-shared-utils/0.1/maven-shared-utils-0.1.jar [ERROR] urls[3] = file:/C:/Users/jianyi/.m2/repository/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar [ERROR] urls[4] = file:/C:/Users/jianyi/.m2/repository/org/codehaus/plexus/plexus-compiler-api/2.4/plexus-compiler-api-2.4.jar [ERROR] urls[5] = file:/C:/Users/jianyi/.m2/repository/org/codehaus/plexus/plexus-compiler-manager/2.4/plexus-compiler-manager-2.4.jar [ERROR] urls[6] = file:/C:/Users/jianyi/.m2/repository/org/codehaus/plexus/plexus-compiler-javac/2.4/plexus-compiler-javac-2.4.jar [ERROR] urls[7] = file:/C:/Users/jianyi/.m2/repository/org/apache/xbean/xbean-reflect/3.4/xbean-reflect-3.4.jar [ERROR] urls[8] = file:/C:/Users/jianyi/.m2/repository/log4j/log4j/1.2.12/log4j-1.2.12.jar [ERROR] urls[9] = file:/C:/Users/jianyi/.m2/repository/commons-logging/commons-logging-api/1.1/commons-logging-api-1.1.jar [ERROR] urls[10] = file:/C:/Users/jianyi/.m2/repository/com/google/collections/google-collections/1.0/google-collections-1.0.jar [ERROR] urls[11] = file:/C:/Users/jianyi/.m2/repository/junit/junit/3.8.2/junit-3.8.2.jar [ERROR] Number of foreign imports: 1 [ERROR] import: Entry[import from realm ClassRealm[project>org.apache.hbase:hbase-thrift:2.0.0-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]] [ERROR] [ERROR] -: org.codehaus.plexus.compiler.util.scan.InclusionScanException [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginContainerException was (Author: eyj...@gmail.com): I compiled it on 1.2.3, I meet compile error on master: ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project hbase-thrift: Execution default-compile of goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile failed: A required class was missing while executing org.apache.maven.plugins:maven-compiler-plugin:3.2:compile: org/codehaus/plexus/compiler/util/scan/InclusionScanException [ERROR] - [ERROR] realm =plugin>org.apache.maven.plugins:maven-compiler-plugin:3.2 [ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy [ERROR] urls[0] = file:/C:/Users/jayyi/.m2/repository/org/apache/maven/plugins/maven-compiler-plugin/3.2/maven-compiler-plugin-3.2.jar [ERROR] urls[1] = file:/C:/Users/jayyi/.m2/repository/org/codehaus/plexus/plexus-utils/1.5.1/plexus-utils-1.5.1.jar [ERROR] urls[2] = file:/C:/Users/jayyi/.m2/repository/org/apache/maven/shared/maven-shared-utils/0.1/maven-shared-utils-0.1.jar [ERROR] urls[3] = file:/C:/Users/jayyi/.m2/repository/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar [ERROR] urls[4] = file:/C:/Users/jayyi/.m2/repository/org/codehaus/plexus/plexus-compiler-api/2.4/plexus-compiler-api-2.4.jar [ERROR] urls[5] = file:/C:/Users/jayyi/.m2/repository/org/codehaus/plexus/plexus-compiler-manager/2.4/plexus-compiler-manager-2.4.jar [ERROR] urls[6] = file:/C:/Users/jayyi/.m2/repository/org/codehaus/plexus/plexus-compiler-javac/2.4/plexus-compiler-javac-2.4.jar [ERROR] urls[7] = file:/C:/Users/jayyi/.m2/repository/org/apache/xbean/xbean-reflect/3.4/xbean-reflect-3.4.jar [ERROR] urls[8] = file:/C:/Users/jayyi/.m2/repository/log4j/log4j/1.2.12/log4j-1.2.12.jar [ERROR]
[jira] [Commented] (HBASE-16561) Add metrics about read/write/scan queue length and active read/write/scan handler count
[ https://issues.apache.org/jira/browse/HBASE-16561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703827#comment-15703827 ] Duo Zhang commented on HBASE-16561: --- +1. Let me help committing the v1 patch. > Add metrics about read/write/scan queue length and active read/write/scan > handler count > --- > > Key: HBASE-16561 > URL: https://issues.apache.org/jira/browse/HBASE-16561 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, metrics >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-16561-v1.patch, HBASE-16561.patch > > > Now there are only metrics about total queue length and active rpc handler > count. But in the RWQueueRpcExecutor, there are different queues and handlers > for read/write/scan request. I thought it is necessary to add more metrics > for RWQueueRpcExecutor. When use it in production cluster, we can adjust the > config of queues and handlers according to the metrics. > Review url: https://reviews.apache.org/r/54072/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17181) Let HBase thrift2 support TThreadedSelectorServer
[ https://issues.apache.org/jira/browse/HBASE-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703825#comment-15703825 ] Jian Yi commented on HBASE-17181: - tested on 1.2.3 version, I compile master unsuccessfully > Let HBase thrift2 support TThreadedSelectorServer > - > > Key: HBASE-17181 > URL: https://issues.apache.org/jira/browse/HBASE-17181 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 1.2.3 >Reporter: Jian Yi >Priority: Minor > Labels: features > Fix For: 1.2.3 > > Attachments: HBASE-17181-V1.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Add TThreadedSelectorServer for HBase Thrift2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17181) Let HBase thrift2 support TThreadedSelectorServer
[ https://issues.apache.org/jira/browse/HBASE-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703823#comment-15703823 ] Jian Yi commented on HBASE-17181: - I compiled it on 1.2.3, I meet compile error on master: ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project hbase-thrift: Execution default-compile of goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile failed: A required class was missing while executing org.apache.maven.plugins:maven-compiler-plugin:3.2:compile: org/codehaus/plexus/compiler/util/scan/InclusionScanException [ERROR] - [ERROR] realm =plugin>org.apache.maven.plugins:maven-compiler-plugin:3.2 [ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy [ERROR] urls[0] = file:/C:/Users/jayyi/.m2/repository/org/apache/maven/plugins/maven-compiler-plugin/3.2/maven-compiler-plugin-3.2.jar [ERROR] urls[1] = file:/C:/Users/jayyi/.m2/repository/org/codehaus/plexus/plexus-utils/1.5.1/plexus-utils-1.5.1.jar [ERROR] urls[2] = file:/C:/Users/jayyi/.m2/repository/org/apache/maven/shared/maven-shared-utils/0.1/maven-shared-utils-0.1.jar [ERROR] urls[3] = file:/C:/Users/jayyi/.m2/repository/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar [ERROR] urls[4] = file:/C:/Users/jayyi/.m2/repository/org/codehaus/plexus/plexus-compiler-api/2.4/plexus-compiler-api-2.4.jar [ERROR] urls[5] = file:/C:/Users/jayyi/.m2/repository/org/codehaus/plexus/plexus-compiler-manager/2.4/plexus-compiler-manager-2.4.jar [ERROR] urls[6] = file:/C:/Users/jayyi/.m2/repository/org/codehaus/plexus/plexus-compiler-javac/2.4/plexus-compiler-javac-2.4.jar [ERROR] urls[7] = file:/C:/Users/jayyi/.m2/repository/org/apache/xbean/xbean-reflect/3.4/xbean-reflect-3.4.jar [ERROR] urls[8] = file:/C:/Users/jayyi/.m2/repository/log4j/log4j/1.2.12/log4j-1.2.12.jar [ERROR] urls[9] = file:/C:/Users/jayyi/.m2/repository/commons-logging/commons-logging-api/1.1/commons-logging-api-1.1.jar [ERROR] urls[10] = file:/C:/Users/jayyi/.m2/repository/com/google/collections/google-collections/1.0/google-collections-1.0.jar [ERROR] urls[11] = file:/C:/Users/jayyi/.m2/repository/junit/junit/3.8.2/junit-3.8.2.jar [ERROR] Number of foreign imports: 1 [ERROR] import: Entry[import from realm ClassRealm[project>org.apache.hbase:hbase-thrift:2.0.0-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]] [ERROR] [ERROR] -: org.codehaus.plexus.compiler.util.scan.InclusionScanException [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginContainerException > Let HBase thrift2 support TThreadedSelectorServer > - > > Key: HBASE-17181 > URL: https://issues.apache.org/jira/browse/HBASE-17181 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 1.2.3 >Reporter: Jian Yi >Priority: Minor > Labels: features > Fix For: 1.2.3 > > Attachments: HBASE-17181-V1.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Add TThreadedSelectorServer for HBase Thrift2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2
[ https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703817#comment-15703817 ] 易剑 commented on HBASE-17182: maybe use the same method with ConnectionCache: ChoreService and ScheduledChore/ScheduledThreadPoolExecutor > Memory leak from openScanner of HBase thrift2 > - > > Key: HBASE-17182 > URL: https://issues.apache.org/jira/browse/HBASE-17182 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: 易剑 > > Client call openScanner, but client (coredump or others) not closeScanner, > the scanner will not be removed from scannerMap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17140) Throw RegionOfflineException directly when request for a disabled table
[
https://issues.apache.org/jira/browse/HBASE-17140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703808#comment-15703808
]
Guanghao Zhang commented on HBASE-17140:
bq. at the moment we only use the offline flag for split/merge. and the regions
may be back if we rollback.
Yeah, the offline flag is only used by split region. But for split region, the
split flag need be true, too. The splited region need two flag, offline and
split. So I thought the disabled region can use the offline flag.
bq. our code has lots of assumptions and changing things from
TableNotEnabledException to RegionOfflineException feels scary to me.
This can be changed back to TableNotEnabledException. This issue's goal is fail
fast when found table is disabled. And don't check table state for any retry
request. Thanks.
> Throw RegionOfflineException directly when request for a disabled table
> ---
>
> Key: HBASE-17140
> URL: https://issues.apache.org/jira/browse/HBASE-17140
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-17140-v1.patch, HBASE-17140-v2.patch,
> HBASE-17140-v3.patch, HBASE-17140-v4.patch, HBASE-17140-v5.patch
>
>
> Now when request for a disabled table, it need 3 rpc calls before fail.
> 1. get region location
> 2. send call to rs and get NotServeRegionException
> 3. retry and check the table state, then throw TableNotEnabledException
> The table state check is added for disabled table. But now the prepare method
> in RegionServerCallable shows that all retry request will get table state
> first.
> {code}
> public void prepare(final boolean reload) throws IOException {
> // check table state if this is a retry
> if (reload && !tableName.equals(TableName.META_TABLE_NAME) &&
> getConnection().isTableDisabled(tableName)) {
> throw new TableNotEnabledException(tableName.getNameAsString() + " is
> disabled.");
> }
> try (RegionLocator regionLocator =
> connection.getRegionLocator(tableName)) {
> this.location = regionLocator.getRegionLocation(row);
> }
> if (this.location == null) {
> throw new IOException("Failed to find location, tableName=" + tableName
> +
> ", row=" + Bytes.toString(row) + ", reload=" + reload);
> }
> setStubByServiceName(this.location.getServerName());
> }
> {code}
> An improvement is set the region offline in HRegionInfo. Then throw the
> RegionOfflineException when get region location.
> Review board: https://reviews.apache.org/r/54071/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17057) Minor compactions should also drop page cache behind reads
[ https://issues.apache.org/jira/browse/HBASE-17057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703800#comment-15703800 ] Ashu Pachauri commented on HBASE-17057: --- I had a first look at the code, things never are straightforward in the land of compactions. 1. Whether we can drop cache behind reads/writes for compaction is tried to whether we should throttle compactions. Not sure what dropping page cache has to do with throttling compactions. [~eclark] , can you shed a little light on why this is the case? (Since you added this piece of code in HBASE-14098 ) 2. It seems that we just ignore drop cache hint for all compactions on master branch. I'll dig more and try to find why the behavior was modified on master. > Minor compactions should also drop page cache behind reads > -- > > Key: HBASE-17057 > URL: https://issues.apache.org/jira/browse/HBASE-17057 > Project: HBase > Issue Type: Improvement > Components: Compaction >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > > Long compactions currently drop cache behind reads so that they don't pollute > the page cache but short compactions don't do that. The bulk of the data is > actually read during minor compactions instead of major compactions, and > thrashes the page cache since it's mostly not needed. > We should drop page cache behind reads for minor compactions too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17128) Find Cause of a Write Perf Regression in branch-1.2
[
https://issues.apache.org/jira/browse/HBASE-17128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703777#comment-15703777
]
Appy commented on HBASE-17128:
--
Hi [~gbaecher], thanks for sharing the details.
[Here's|https://docs.google.com/a/cloudera.com/document/d/e/2PACX-1vSA9iZ9rtBvQgxm02vkaA1dL6P-cz6DVxOtq1U8VweyEjR5mfYfTvdEwUohJV1992mrxGzgZXgamBbX/pub]
the rough doc i was keeping when running tests.
bq. we tested both CDH 5.7 and 5.9
Those versions refer to version of the cluster or the client? Also, it'd be
great to have numbers for this too.
bq. We tested against the CDH 5.9 server code, with both CDH 5.4.5 and CDH 5.9
client code.
How exactly are you running with different clients? Afaik, ycsb has its own
bindings (hbase10, hbase098, etc). By cdh5.4.5 client, do you mean you are
using hbase098 binding and correspondingly, does cdh5.9 client means hbase10
binding? Maybe the exact commands will clear up things.
> Find Cause of a Write Perf Regression in branch-1.2
> ---
>
> Key: HBASE-17128
> URL: https://issues.apache.org/jira/browse/HBASE-17128
> Project: HBase
> Issue Type: Task
>Reporter: stack
>
> As reported by [~gbaecher] up on the mailing list, there is a regression in
> 1.2. The regression is in a CDH version of 1.2 actually but the CDH hbase is
> a near pure 1.2. This is a working issue to figure which of the below changes
> brought on slower writes (The list comes from doing the following...git log
> --oneline
> remotes/origin/cdh5-1.2.0_5.8.0_dev..remotes/origin/cdh5-1.2.0_5.9.0_dev ...
> I stripped the few CDH specific changes, packaging and tagging only, and then
> made two groupings; candidates and the unlikelies):
> {code}
> 1 bbc6762 HBASE-16023 Fastpath for the FIFO rpcscheduler Adds an executor
> that does balanced queue and fast path handing off requests directly to
> waiting handlers if any present. Idea taken from Apace Kudu (incubating). See
> https://gerr#
> 2 a260917 HBASE-16288 HFile intermediate block level indexes might recurse
> forever creating multi TB files
> 3 5633281 HBASE-15811 Batch Get after batch Put does not fetch all Cells We
> were not waiting on all executors in a batch to complete. The test for
> no-more-executors was damaged by the 0.99/0.98.4 fix "HBASE-11403 Fix race
> conditions aro#
> 4 780f720 HBASE-11625 - Verifies data before building HFileBlock. - Adds
> HFileBlock.Header class which contains information about location of fields.
> Testing: Adds CorruptedFSReaderImpl to TestChecksum. (Apekshit)
> 5 d735680 HBASE-12133 Add FastLongHistogram for metric computation (Yi Deng)
> 6 c4ee832 HBASE-15222 Use less contended classes for metrics
> 7
> 8 17320a4 HBASE-15683 Min latency in latency histograms are emitted as
> Long.MAX_VALUE
> 9 283b39f HBASE-15396 Enhance mapreduce.TableSplit to add encoded region
> name
> 10 39db592 HBASE-16195 Should not add chunk into chunkQueue if not using
> chunk pool in HeapMemStoreLAB
> 11 5ff28b7 HBASE-16194 Should count in MSLAB chunk allocation into heap size
> change when adding duplicate cells
> 12 5e3e0d2 HBASE-16318 fail build while rendering velocity template if
> dependency license isn't in whitelist.
> 13 3ed66e3 HBASE-16318 consistently use the correct name for 'Apache
> License, Version 2.0'
> 14 351832d HBASE-16340 exclude Xerces iplementation jars from coming in
> transitively.
> 15 b6aa4be HBASE-16321 ensure no findbugs-jsr305
> 16 4f9dde7 HBASE-16317 revert all ESAPI changes
> 17 71b6a8a HBASE-16284 Unauthorized client can shutdown the cluster (Deokwoo
> Han)
> 18 523753f HBASE-16450 Shell tool to dump replication queues
> 19 ca5f2ee HBASE-16379 [replication] Minor improvement to
> replication/copy_tables_desc.rb
> 20 effd105 HBASE-16135 PeerClusterZnode under rs of removed peer may never
> be deleted
> 21 a5c6610 HBASE-16319 Fix TestCacheOnWrite after HBASE-16288
> 22 1956bb0 HBASE-15808 Reduce potential bulk load intermediate space usage
> and waste
> 23 031c54e HBASE-16096 Backport. Cleanly remove replication peers from
> ZooKeeper.
> 24 60a3b12 HBASE-14963 Remove use of Guava Stopwatch from HBase client code
> (Devaraj Das)
> 25 c7724fc HBASE-16207 can't restore snapshot without "Admin" permission
> 26 8322a0b HBASE-16227 [Shell] Column value formatter not working in scans.
> Tested : manually using shell.
> 27 8f86658 HBASE-14818 user_permission does not list namespace permissions
> (li xiang)
> 28 775cd21 HBASE-15465 userPermission returned by getUserPermission() for
> the selected namespace does not have namespace set (li xiang)
> 29 8d85aff HBASE-16093 Fix splits failed before creating daughter regions
> leave meta inconsistent
> 30 bc41317 HBASE-16140 bump owasp.esapi from 2.1.0 to 2.1.0.1
> 31 6fc70cd HBASE-16035 Nested AutoCloseables might
[jira] [Updated] (HBASE-17185) Purge the seek of the next block reading HFileBlocks
[ https://issues.apache.org/jira/browse/HBASE-17185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17185: -- Fix Version/s: 2.0.0 Status: Patch Available (was: Open) > Purge the seek of the next block reading HFileBlocks > > > Key: HBASE-17185 > URL: https://issues.apache.org/jira/browse/HBASE-17185 > Project: HBase > Issue Type: Improvement > Components: HFile >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-17185.master.001.patch, HBASE-17185.patch > > > When we read HFileBlocks, we read the asked-for block AND the next block's > header which we add to a cache (see HBASE-17072). We do this extra read to > get the next block's length purportedly. This seek of the next block's header > complicates the HFileBlock construction (not to mind other consequences -- > again see HBASE-17072). > Study done in HBASE-17072 shows that we normally do not need this extra read > of the next block's header. In the usual case, the length of the block is > gotten from the hfile index. > A simplification of block reading can be done purging this extra header read. > We can also save some space in cache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17185) Purge the seek of the next block reading HFileBlocks
[ https://issues.apache.org/jira/browse/HBASE-17185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17185: -- Attachment: HBASE-17185.master.001.patch > Purge the seek of the next block reading HFileBlocks > > > Key: HBASE-17185 > URL: https://issues.apache.org/jira/browse/HBASE-17185 > Project: HBase > Issue Type: Improvement > Components: HFile >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Attachments: HBASE-17185.master.001.patch, HBASE-17185.patch > > > When we read HFileBlocks, we read the asked-for block AND the next block's > header which we add to a cache (see HBASE-17072). We do this extra read to > get the next block's length purportedly. This seek of the next block's header > complicates the HFileBlock construction (not to mind other consequences -- > again see HBASE-17072). > Study done in HBASE-17072 shows that we normally do not need this extra read > of the next block's header. In the usual case, the length of the block is > gotten from the hfile index. > A simplification of block reading can be done purging this extra header read. > We can also save some space in cache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17114) Add an option to set special retry pause when encountering CallQueueTooBigException
[
https://issues.apache.org/jira/browse/HBASE-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703756#comment-15703756
]
Gary Helmling commented on HBASE-17114:
---
+1 on patch v2.
Please add an entry for this new config, with an empty value but with a
description, to hbase-default.xml.
I still think this test is going to be flaky on highly loaded build machines
(like ASF infra).
> Add an option to set special retry pause when encountering
> CallQueueTooBigException
> ---
>
> Key: HBASE-17114
> URL: https://issues.apache.org/jira/browse/HBASE-17114
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-17114.patch, HBASE-17114.v2.patch
>
>
> As titled, after HBASE-15146 we will throw {{CallQueueTooBigException}}
> instead of dead-wait. This is good for performance for most cases but might
> cause a side-effect that if too many clients connect to the busy RS, that the
> retry requests may come over and over again and RS never got the chance for
> recovering, and the issue will become especially critical when the target
> region is META.
> So here in this JIRA we propose to supply some special retry pause for CQTBE
> in name of {{hbase.client.pause.special}}, and by default it will be 500ms (5
> times of {{hbase.client.pause}} default value)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16912) TEST: update HBaseTestingUtility to avoid direct use of filesystem / legacy implementation
[
https://issues.apache.org/jira/browse/HBASE-16912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703748#comment-15703748
]
Hadoop QA commented on HBASE-16912:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 40 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
32s {color} | {color:green} hbase-14439 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} hbase-14439 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s {color} | {color:green} hbase-14439 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
31s {color} | {color:green} hbase-14439 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 43s
{color} | {color:red} hbase-server in hbase-14439 has 5 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} hbase-14439 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 6s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} hbase-server generated 0 new + 8 unchanged - 3 fixed =
8 total (was 11) {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 22s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 61m 44s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestBulkLoad |
| | hadoop.hbase.regionserver.TestDateTieredCompactionPolicy |
| | hadoop.hbase.master.snapshot.TestSnapshotManager |
| | hadoop.hbase.regionserver.TestSplitTransaction |
| | hadoop.hbase.snapshot.TestRestoreSnapshotHelper |
| | hadoop.hbase.snapshot.TestMobRestoreSnapshotHelper |
| | hadoop.hbase.regionserver.TestStoreFileRefresherChore |
| | hadoop.hbase.fs.legacy.snapshot.TestSnapshotHFileCleaner |
| | hadoop.hbase.regionserver.TestDateTieredCompactionPolicyOverflow |
| | hadoop.hbase.master.TestCatalogJanitor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12840755/HBASE-16912.hbase-14439.003.patch
|
| JIRA Issue | HBASE-16912 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 40b9337e6808 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | hbase-14439 / a36f80c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
[jira] [Commented] (HBASE-17186) MasterProcedureTestingUtility#testRecoveryAndDoubleExecution displays stale procedure state info
[
https://issues.apache.org/jira/browse/HBASE-17186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703735#comment-15703735
]
Hadoop QA commented on HBASE-17186:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 21s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 96m 53s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 134m 14s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12840736/HBASE-17186.v1-master.patch
|
| JIRA Issue | HBASE-17186 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 37a65d37ba93 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 3d5e686 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4655/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4655/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> MasterProcedureTestingUtility#testRecoveryAndDoubleExecution displays stale
> procedure state info
>
>
> Key: HBASE-17186
> URL: https://issues.apache.org/jira/browse/HBASE-17186
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, test
>Reporter: Stephen Yuan Jiang
>Assignee: Stephen Yuan
[jira] [Assigned] (HBASE-17057) Minor compactions should also drop page cache behind reads
[ https://issues.apache.org/jira/browse/HBASE-17057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashu Pachauri reassigned HBASE-17057: - Assignee: Ashu Pachauri > Minor compactions should also drop page cache behind reads > -- > > Key: HBASE-17057 > URL: https://issues.apache.org/jira/browse/HBASE-17057 > Project: HBase > Issue Type: Improvement > Components: Compaction >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > > Long compactions currently drop cache behind reads so that they don't pollute > the page cache but short compactions don't do that. The bulk of the data is > actually read during minor compactions instead of major compactions, and > thrashes the page cache since it's mostly not needed. > We should drop page cache behind reads for minor compactions too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17189) TestMasterObserver#wasModifyTableActionCalled uses wrong variables
[
https://issues.apache.org/jira/browse/HBASE-17189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703712#comment-15703712
]
Matteo Bertozzi commented on HBASE-17189:
-
+1
> TestMasterObserver#wasModifyTableActionCalled uses wrong variables
> --
>
> Key: HBASE-17189
> URL: https://issues.apache.org/jira/browse/HBASE-17189
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Stephen Yuan Jiang
>Assignee: Stephen Yuan Jiang
>Priority: Minor
> Attachments: HBASE-17189.v1-master.patch
>
>
> TestMasterObserver#wasModifyTableActionCalled() and
> TestMasterObserver#wasModifyTableActionCalledOnly() uses
> {{preModifyColumnFamilyActionCalled}} and
> {{postCompletedModifyColumnFamilyActionCalled}} members, which are wrong.
> Instead it should use {{preModifyTableActionCalled}} and
> {{postCompletedModifyTableActionCalled}}. This probably was caused by
> copy-and-paste mistake.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-17189) TestMasterObserver#wasModifyTableActionCalled uses wrong variables
[
https://issues.apache.org/jira/browse/HBASE-17189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Yuan Jiang updated HBASE-17189:
---
Status: Patch Available (was: Open)
> TestMasterObserver#wasModifyTableActionCalled uses wrong variables
> --
>
> Key: HBASE-17189
> URL: https://issues.apache.org/jira/browse/HBASE-17189
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Stephen Yuan Jiang
>Assignee: Stephen Yuan Jiang
>Priority: Minor
> Attachments: HBASE-17189.v1-master.patch
>
>
> TestMasterObserver#wasModifyTableActionCalled() and
> TestMasterObserver#wasModifyTableActionCalledOnly() uses
> {{preModifyColumnFamilyActionCalled}} and
> {{postCompletedModifyColumnFamilyActionCalled}} members, which are wrong.
> Instead it should use {{preModifyTableActionCalled}} and
> {{postCompletedModifyTableActionCalled}}. This probably was caused by
> copy-and-paste mistake.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-17189) TestMasterObserver#wasModifyTableActionCalled uses wrong variables
[
https://issues.apache.org/jira/browse/HBASE-17189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Yuan Jiang updated HBASE-17189:
---
Attachment: HBASE-17189.v1-master.patch
> TestMasterObserver#wasModifyTableActionCalled uses wrong variables
> --
>
> Key: HBASE-17189
> URL: https://issues.apache.org/jira/browse/HBASE-17189
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Stephen Yuan Jiang
>Assignee: Stephen Yuan Jiang
>Priority: Minor
> Attachments: HBASE-17189.v1-master.patch
>
>
> TestMasterObserver#wasModifyTableActionCalled() and
> TestMasterObserver#wasModifyTableActionCalledOnly() uses
> {{preModifyColumnFamilyActionCalled}} and
> {{postCompletedModifyColumnFamilyActionCalled}} members, which are wrong.
> Instead it should use {{preModifyTableActionCalled}} and
> {{postCompletedModifyTableActionCalled}}. This probably was caused by
> copy-and-paste mistake.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-17189) TestMasterObserver#wasModifyTableActionCalled uses wrong variables
Stephen Yuan Jiang created HBASE-17189:
--
Summary: TestMasterObserver#wasModifyTableActionCalled uses wrong
variables
Key: HBASE-17189
URL: https://issues.apache.org/jira/browse/HBASE-17189
Project: HBase
Issue Type: Test
Components: test
Reporter: Stephen Yuan Jiang
Assignee: Stephen Yuan Jiang
Priority: Minor
TestMasterObserver#wasModifyTableActionCalled() and
TestMasterObserver#wasModifyTableActionCalledOnly() uses
{{preModifyColumnFamilyActionCalled}} and
{{postCompletedModifyColumnFamilyActionCalled}} members, which are wrong.
Instead it should use {{preModifyTableActionCalled}} and
{{postCompletedModifyTableActionCalled}}. This probably was caused by
copy-and-paste mistake.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17128) Find Cause of a Write Perf Regression in branch-1.2
[
https://issues.apache.org/jira/browse/HBASE-17128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703646#comment-15703646
]
Graham Baecher commented on HBASE-17128:
Thanks stack.
We noticed these today: https://issues.apache.org/jira/browse/HBASE-16616 and
https://issues.apache.org/jira/browse/HBASE-16146
Both seem potentially relevant. HBASE-16616 specifically mentions that G1GC
isn't properly cleaning up some counters if they survive into Tenured space. We
are using G1GC for these servers--can try some tests on our end to see if the
regression goes away with a different garbage collector or with some
metrics-related commits removed.
> Find Cause of a Write Perf Regression in branch-1.2
> ---
>
> Key: HBASE-17128
> URL: https://issues.apache.org/jira/browse/HBASE-17128
> Project: HBase
> Issue Type: Task
>Reporter: stack
>
> As reported by [~gbaecher] up on the mailing list, there is a regression in
> 1.2. The regression is in a CDH version of 1.2 actually but the CDH hbase is
> a near pure 1.2. This is a working issue to figure which of the below changes
> brought on slower writes (The list comes from doing the following...git log
> --oneline
> remotes/origin/cdh5-1.2.0_5.8.0_dev..remotes/origin/cdh5-1.2.0_5.9.0_dev ...
> I stripped the few CDH specific changes, packaging and tagging only, and then
> made two groupings; candidates and the unlikelies):
> {code}
> 1 bbc6762 HBASE-16023 Fastpath for the FIFO rpcscheduler Adds an executor
> that does balanced queue and fast path handing off requests directly to
> waiting handlers if any present. Idea taken from Apace Kudu (incubating). See
> https://gerr#
> 2 a260917 HBASE-16288 HFile intermediate block level indexes might recurse
> forever creating multi TB files
> 3 5633281 HBASE-15811 Batch Get after batch Put does not fetch all Cells We
> were not waiting on all executors in a batch to complete. The test for
> no-more-executors was damaged by the 0.99/0.98.4 fix "HBASE-11403 Fix race
> conditions aro#
> 4 780f720 HBASE-11625 - Verifies data before building HFileBlock. - Adds
> HFileBlock.Header class which contains information about location of fields.
> Testing: Adds CorruptedFSReaderImpl to TestChecksum. (Apekshit)
> 5 d735680 HBASE-12133 Add FastLongHistogram for metric computation (Yi Deng)
> 6 c4ee832 HBASE-15222 Use less contended classes for metrics
> 7
> 8 17320a4 HBASE-15683 Min latency in latency histograms are emitted as
> Long.MAX_VALUE
> 9 283b39f HBASE-15396 Enhance mapreduce.TableSplit to add encoded region
> name
> 10 39db592 HBASE-16195 Should not add chunk into chunkQueue if not using
> chunk pool in HeapMemStoreLAB
> 11 5ff28b7 HBASE-16194 Should count in MSLAB chunk allocation into heap size
> change when adding duplicate cells
> 12 5e3e0d2 HBASE-16318 fail build while rendering velocity template if
> dependency license isn't in whitelist.
> 13 3ed66e3 HBASE-16318 consistently use the correct name for 'Apache
> License, Version 2.0'
> 14 351832d HBASE-16340 exclude Xerces iplementation jars from coming in
> transitively.
> 15 b6aa4be HBASE-16321 ensure no findbugs-jsr305
> 16 4f9dde7 HBASE-16317 revert all ESAPI changes
> 17 71b6a8a HBASE-16284 Unauthorized client can shutdown the cluster (Deokwoo
> Han)
> 18 523753f HBASE-16450 Shell tool to dump replication queues
> 19 ca5f2ee HBASE-16379 [replication] Minor improvement to
> replication/copy_tables_desc.rb
> 20 effd105 HBASE-16135 PeerClusterZnode under rs of removed peer may never
> be deleted
> 21 a5c6610 HBASE-16319 Fix TestCacheOnWrite after HBASE-16288
> 22 1956bb0 HBASE-15808 Reduce potential bulk load intermediate space usage
> and waste
> 23 031c54e HBASE-16096 Backport. Cleanly remove replication peers from
> ZooKeeper.
> 24 60a3b12 HBASE-14963 Remove use of Guava Stopwatch from HBase client code
> (Devaraj Das)
> 25 c7724fc HBASE-16207 can't restore snapshot without "Admin" permission
> 26 8322a0b HBASE-16227 [Shell] Column value formatter not working in scans.
> Tested : manually using shell.
> 27 8f86658 HBASE-14818 user_permission does not list namespace permissions
> (li xiang)
> 28 775cd21 HBASE-15465 userPermission returned by getUserPermission() for
> the selected namespace does not have namespace set (li xiang)
> 29 8d85aff HBASE-16093 Fix splits failed before creating daughter regions
> leave meta inconsistent
> 30 bc41317 HBASE-16140 bump owasp.esapi from 2.1.0 to 2.1.0.1
> 31 6fc70cd HBASE-16035 Nested AutoCloseables might not all get closed (Sean
> Mackrory)
> 32 fe28fe84 HBASE-15891. Closeable resources potentially not getting closed
> if exception is thrown.
> 33 1d2bf3c HBASE-14644 Region in transition metric is broken -- addendum
> (Huaxiang Sun)
> 34 fd5f56c HBASE-16056 Procedure v2
[jira] [Comment Edited] (HBASE-17128) Find Cause of a Write Perf Regression in branch-1.2
[
https://issues.apache.org/jira/browse/HBASE-17128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703483#comment-15703483
]
Graham Baecher edited comment on HBASE-17128 at 11/29/16 12:08 AM:
---
Our test cluster is set up in AWS. As far as hardware specs, we're running 4
RegionServers on
[d2.xlarge|http://www.ec2instances.info/?selected=d2.xlarge#d2.xlarge]
instances. Our YCSB workload is running on a
[c3.4xlarge|http://www.ec2instances.info/?selected=c3.4xlarge#c3.4xlarge], with
500 threads for the workload runs.
As far as configs, the RegionServers run on 25GB heaps, pretty standard
configs, though I as I mentioned on the user list, we tested both CDH 5.7 and
5.9 with deadline callqueue instead of FIFO. We also have
{{hbase.regionserver.wal.enablecompression}} set to true.
For handler threads, we have:
- {{hbase.regionserver.handler.count}} = 50
- {{hbase.ipc.server.callqueue.handler.factor}} = 0.3
- {{hbase.ipc.server.callqueue.read.ratio}} = 0.5
- {{hbase.ipc.server.callqueue.scan.ratio}} = 0.5
We tested against the CDH 5.9 server code, with both CDH 5.4.5 and CDH 5.9
client code. With the CDH 5.9 client, workload A was actually slightly slower
than with CDH 5.4.5. I did notice that CDH 5.9's reads were slower than CDH
5.4.5's while CDH 5.4.5's writes were slower than CDH 5.9's, which seems to
match Appy's findings.
For reference, on the setup above, here's what we're seeing for workload A:
|(CDH 5.9 servers) ||CDH 5.4.5 client||CDH 5.9 client||
|Read mean latency|~4ms|~8ms|
|Write mean latency|~28ms|~26ms|
With a 50/50 read/write ratio, both of these are net regressions from CDH 5.8
server code:
running this workload with the CDH 5.4.5 client against CDH 5.8 RegionServers
had averages slightly over 5 ms for reads and around 22 ms for writes, so
running workload A against CDH 5.8 was much faster for us overall.
was (Author: gbaecher):
Our test cluster is set up in AWS. As far as hardware specs, we're running 4
RegionServers on
[d2.xlarge|http://www.ec2instances.info/?selected=d2.xlarge#d2.xlarge]
instances. Our YCSB workload is running on a
[c3.4xlarge|http://www.ec2instances.info/?selected=c3.4xlarge#c3.4xlarge], with
500 threads for the workload runs.
As far as configs, the RegionServers run on 25GB heaps, pretty standard
configs, though I as I mentioned on the user list, we tested both CDH 5.7 and
5.9 with deadline callqueue instead of FIFO. We also have
{{hbase.regionserver.wal.enablecompression}} set to true.
For handler threads, we have:
- {{hbase.regionserver.handler.count}} = 50
- {{hbase.ipc.server.callqueue.handler.factor}} = 0.3
- {{hbase.ipc.server.callqueue.read.ratio}} = 0.5
- {{hbase.ipc.server.callqueue.scan.ratio}} = 0.5
We tested against the CDH 5.9 server code, with both CDH 5.4.5 and CDH 5.9
client code. With the CDH 5.9 client, workload A was actually slightly slower
than with CDH 5.4.5. I did notice that CDH 5.9's reads were slower than CDH
5.4.5's while CDH 5.4.5's writes were slower than CDH 5.9's, which seems to
match Appy's findings.
For reference, on the setup above, here's what we're seeing for workload A:
| ||CDH 5.4.5 client||CDH 5.9 client||
|Read mean latency|~4ms|~8ms|
|Write mean latency|~28ms|~26ms|
Previously, running this workload with the CDH 5.4.5 client against CDH 5.8
RegionServers had averages slightly over 5 ms for reads and around 22 ms for
writes, so it was much faster overall.
> Find Cause of a Write Perf Regression in branch-1.2
> ---
>
> Key: HBASE-17128
> URL: https://issues.apache.org/jira/browse/HBASE-17128
> Project: HBase
> Issue Type: Task
>Reporter: stack
>
> As reported by [~gbaecher] up on the mailing list, there is a regression in
> 1.2. The regression is in a CDH version of 1.2 actually but the CDH hbase is
> a near pure 1.2. This is a working issue to figure which of the below changes
> brought on slower writes (The list comes from doing the following...git log
> --oneline
> remotes/origin/cdh5-1.2.0_5.8.0_dev..remotes/origin/cdh5-1.2.0_5.9.0_dev ...
> I stripped the few CDH specific changes, packaging and tagging only, and then
> made two groupings; candidates and the unlikelies):
> {code}
> 1 bbc6762 HBASE-16023 Fastpath for the FIFO rpcscheduler Adds an executor
> that does balanced queue and fast path handing off requests directly to
> waiting handlers if any present. Idea taken from Apace Kudu (incubating). See
> https://gerr#
> 2 a260917 HBASE-16288 HFile intermediate block level indexes might recurse
> forever creating multi TB files
> 3 5633281 HBASE-15811 Batch Get after batch Put does not fetch all Cells We
> were not waiting on all executors in a batch to complete. The test for
> no-more-executors was damaged by the 0.99/0.98.4 fix "HBASE-11403 Fix race
>
[jira] [Commented] (HBASE-17154) Backup progress command usage small fix
[
https://issues.apache.org/jira/browse/HBASE-17154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703641#comment-15703641
]
Hadoop QA commented on HBASE-17154:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 3s {color}
| {color:red} HBASE-17154 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12840756/HBASE-17154-v1.patch |
| JIRA Issue | HBASE-17154 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4658/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Backup progress command usage small fix
>
>
> Key: HBASE-17154
> URL: https://issues.apache.org/jira/browse/HBASE-17154
> Project: HBase
> Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-17154-v1.patch
>
>
> Usage should describe that backupId is optional and command w/o backup id
> shows progress of a current backup session.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17178) Add region balance throttling
[ https://issues.apache.org/jira/browse/HBASE-17178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703628#comment-15703628 ] Guanghao Zhang commented on HBASE-17178: Thanks for your help. The goal of HBASE-12890 is to limit the total number regions in one balance round. And this issue's goal is to throttle the balance speed. > Add region balance throttling > - > > Key: HBASE-17178 > URL: https://issues.apache.org/jira/browse/HBASE-17178 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17178-v1.patch > > > Our online cluster serves dozens of tables and different tables serve for > different services. If the balancer moves too many regions in the same time, > it will decrease the availability for some table or some services. So we add > region balance throttling on our online serve cluster. > We introduce a new config hbase.balancer.max.balancing.regions, which means > the max number of regions in transition when balancing. > If we config this to 1 and a table have 100 regions, then the table will have > 99 regions available at any time. It helps a lot for our use case and it has > been running a long time > our production cluster. > But for some use case, we need the balancer run faster. If a cluster has 100 > regionservers, then it add 50 new regionservers for peak requests. Then it > need balancer run as soon as > possible and let the cluster reach a balance state soon. Our idea is compute > max number of regions in transition by the max balancing time and the average > time of region in transition. > Then the balancer use the computed value to throttling. > Examples for understanding. > A cluster has 100 regionservers, each regionserver has 200 regions and the > average time of region in transition is 1 seconds, we config the max > balancing time is 10 * 60 seconds. > Case 1. One regionserver crash, the cluster at most need balance 200 regions. > Then 200 / (10 * 60s / 1s) < 1, it means the max number of regions in > transition is 1 when balancing. Then the balancer can move region one by one > and the cluster will have high availability when balancing. > Case 2. Add other 100 regionservers, the cluster at most need balance 1 > regions. Then 1 / (10 * 60s / 1s) = 16.7, it means the max number of > regions in transition is 17 when balancing. Then the cluster can reach a > balance state within the max balancing time. > Any suggestions are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
