[jira] [Commented] (HBASE-14918) In-Memory MemStore Flush and Compaction
[ https://issues.apache.org/jira/browse/HBASE-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166882#comment-15166882 ] ramkrishna.s.vasudevan commented on HBASE-14918: bq.Yeah, lets align what you are doing here with the offheaping of the write path work +1 here. Was waiting for Anoop to reply over here. The MSLAB should not be working with ByteRange or its forms rather it should be with a datastructure that can work with offheap also. So ByteBuffer is the ideal choice here. bq.Should base type be ByteBuff so can do onheap/offheap? Bytebuffs are wrappers on Bytebbuffers so unless we need something like multiple buffers we need not go with ByteBuffs. > In-Memory MemStore Flush and Compaction > --- > > Key: HBASE-14918 > URL: https://issues.apache.org/jira/browse/HBASE-14918 > Project: HBase > Issue Type: Umbrella >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Fix For: 0.98.18 > > Attachments: CellBlocksSegmentDesign.pdf, MSLABMove.patch > > > A memstore serves as the in-memory component of a store unit, absorbing all > updates to the store. From time to time these updates are flushed to a file > on disk, where they are compacted (by eliminating redundancies) and > compressed (i.e., written in a compressed format to reduce their storage > size). > We aim to speed up data access, and therefore suggest to apply in-memory > memstore flush. That is to flush the active in-memory segment into an > intermediate buffer where it can be accessed by the application. Data in the > buffer is subject to compaction and can be stored in any format that allows > it to take up smaller space in RAM. The less space the buffer consumes the > longer it can reside in memory before data is flushed to disk, resulting in > better performance. > Specifically, the optimization is beneficial for workloads with > medium-to-high key churn which incur many redundant cells, like persistent > messaging. > We suggest to structure the solution as 4 subtasks (respectively, patches). > (1) Infrastructure - refactoring of the MemStore hierarchy, introducing > segment (StoreSegment) as first-class citizen, and decoupling memstore > scanner from the memstore implementation; > (2) Adding StoreServices facility at the region level to allow memstores > update region counters and access region level synchronization mechanism; > (3) Implementation of a new memstore (CompactingMemstore) with non-optimized > immutable segment representation, and > (4) Memory optimization including compressed format representation and off > heap allocations. > This Jira continues the discussion in HBASE-13408. > Design documents, evaluation results and previous patches can be found in > HBASE-13408. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9393) Hbase does not closing a closed socket resulting in many CLOSE_WAIT
[
https://issues.apache.org/jira/browse/HBASE-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166880#comment-15166880
]
Ashish Singhi commented on HBASE-9393:
--
Thanks for the comments.
{quote}
Several of your log messages are at ERROR but they don't give an operator any
idea about what to do next. No info on finding a root cause, something to
correct, or get more information. Could we add some of that kind of information
or change them to be INFO or WARN?
{quote}
I will change the level and point to this jira in the log message.
bq. Why are we making these assignments indirectly via methods?
To handle the findbugs warnings {{ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD:
Write to static field from instance method}}.
bq. This doesn't look like it will behave correctly in presence of concurrency.
Can we do the reflection set up during a static initializer?
Intial plan was to do that way and do this set up only once (which we still do
only once in the patch) but then the stream is a instance object so could not
do that way. Regarding concurrency we have handled that, the parallel reads
will come through HFileBlock reads and there we acquiring the stream lock and
then call the stream unbuffer, same as we do the for the block reads.
bq. This should probably have an else clause that similarly gives the warning.
In that case, it should probably give a pointer to this issue.
Will address it.
> Hbase does not closing a closed socket resulting in many CLOSE_WAIT
>
>
> Key: HBASE-9393
> URL: https://issues.apache.org/jira/browse/HBASE-9393
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.2, 0.98.0
> Environment: Centos 6.4 - 7 regionservers/datanodes, 8 TB per node,
> 7279 regions
>Reporter: Avi Zrachya
>Assignee: Ashish Singhi
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-9393.patch, HBASE-9393.v1.patch,
> HBASE-9393.v10.patch, HBASE-9393.v11.patch, HBASE-9393.v12.patch,
> HBASE-9393.v2.patch, HBASE-9393.v3.patch, HBASE-9393.v4.patch,
> HBASE-9393.v5.patch, HBASE-9393.v5.patch, HBASE-9393.v5.patch,
> HBASE-9393.v6.patch, HBASE-9393.v6.patch, HBASE-9393.v6.patch,
> HBASE-9393.v7.patch, HBASE-9393.v8.patch, HBASE-9393.v9.patch
>
>
> HBase dose not close a dead connection with the datanode.
> This resulting in over 60K CLOSE_WAIT and at some point HBase can not connect
> to the datanode because too many mapped sockets from one host to another on
> the same port.
> The example below is with low CLOSE_WAIT count because we had to restart
> hbase to solve the porblem, later in time it will incease to 60-100K sockets
> on CLOSE_WAIT
> [root@hd2-region3 ~]# netstat -nap |grep CLOSE_WAIT |grep 21592 |wc -l
> 13156
> [root@hd2-region3 ~]# ps -ef |grep 21592
> root 17255 17219 0 12:26 pts/000:00:00 grep 21592
> hbase21592 1 17 Aug29 ?03:29:06
> /usr/java/jdk1.6.0_26/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Xmx8000m
> -ea -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode
> -Dhbase.log.dir=/var/log/hbase
> -Dhbase.log.file=hbase-hbase-regionserver-hd2-region3.swnet.corp.log ...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15324) Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy and trigger unexpected split
[
https://issues.apache.org/jira/browse/HBASE-15324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166879#comment-15166879
]
Anoop Sam John commented on HBASE-15324:
Can this be done with DisabledRegionSplitPolicy?
> Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy
> and trigger unexpected split
> --
>
> Key: HBASE-15324
> URL: https://issues.apache.org/jira/browse/HBASE-15324
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.1.3
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-15324.patch, HBASE-15324_v2.patch
>
>
> We introduce jitter for region split decision in HBASE-13412, but the
> following line in {{ConstantSizeRegionSplitPolicy}} may cause long value
> overflow if MAX_FILESIZE is specified to Long.MAX_VALUE:
> {code}
> this.desiredMaxFileSize += (long)(desiredMaxFileSize * (RANDOM.nextFloat() -
> 0.5D) * jitter);
> {code}
> In our case we specify MAX_FILESIZE to Long.MAX_VALUE to prevent target
> region to split.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15222) Use less contended classes for metrics
[ https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166870#comment-15166870 ] wangdawei commented on HBASE-15222: --- 你是中国人吗?不好意思，英语不好，想请教关于hbase的两个问题： 1.我使用MR(put方式)建二级索引表，但是Job运行时间很长，最后直接reduce直接卡在reduce 99%，查看reduce task，发现写入不进去了，使用jstack -gcutil 查看YARNCHILD没有full gc,查看HRegionServer(Xms4G)一直full gc，请问你有什么好的建议吗？ 2.有一RS的日志总是输出：INFO org.apache.hadoop.hbase.regionser.HRegionServer:SCanner 51877 lease expired on region tablename,145377609333.44a5b5020f...,请问怎么解决或者好的建议？ 谢谢！！方便的话请加我QQ:1736333563，或者告诉我你的联系方式 > Use less contended classes for metrics > -- > > Key: HBASE-15222 > URL: https://issues.apache.org/jira/browse/HBASE-15222 > Project: HBase > Issue Type: Bug > Components: metrics >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15222-v1.patch, HBASE-15222-v10.patch, > HBASE-15222-v11.patch, HBASE-15222-v12.patch, HBASE-15222-v13.patch, > HBASE-15222-v2.patch, HBASE-15222-v3.patch, HBASE-15222-v5.patch, > HBASE-15222-v6.patch, HBASE-15222-v8.patch, HBASE-15222-v9.patch, > HBASE-15222.patch > > > Running the benchmarks now, but it looks like the results are pretty extreme. > The locking in our histograms is pretty extreme. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166864#comment-15166864 ] Hudson commented on HBASE-15264: FAILURE: Integrated in HBase-Trunk_matrix #737 (See [https://builds.apache.org/job/HBase-Trunk_matrix/737/]) HBASE-15264 Implement a fan out HDFS OutputStream (zhangduo: rev 6e9d355b12a1e666f4d05be02775a01b6754d063) * hbase-server/src/test/java/org/apache/hadoop/hbase/util/FanOutOneBlockAsyncDFSOutputFlushHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/FanOutOneBlockAsyncDFSOutputHelper.java * hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestFanOutOneBlockAsyncDFSOutput.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/FanOutOneBlockAsyncDFSOutput.java > Implement a fan out HDFS OutputStream > - > > Key: HBASE-15264 > URL: https://issues.apache.org/jira/browse/HBASE-15264 > Project: HBase > Issue Type: Sub-task > Components: util, wal >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-15264-v1.patch, HBASE-15264-v2.patch, > HBASE-15264-v3.patch, HBASE-15264-v4.patch, HBASE-15264-v5.patch, > HBASE-15264-v6.patch, HBASE-15264-v7.patch, HBASE-15264-v8.patch, > HBASE-15264.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166863#comment-15166863 ] Hudson commented on HBASE-15136: FAILURE: Integrated in HBase-Trunk_matrix #737 (See [https://builds.apache.org/job/HBase-Trunk_matrix/737/]) HBASE-15136 Explore different queuing behaviors while busy (antonov: rev 43f99def670551cfe314c44181c0cb9570cdaaa3) * hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerWrapperStub.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/FifoRpcScheduler.java * hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestSimpleRpcScheduler.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSource.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerWrapperImpl.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/ReflectionUtils.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RWQueueRpcExecutor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcScheduler.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerWrapper.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/AdaptiveLifoCoDelCallQueue.java > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15324) Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy and trigger unexpected split
[
https://issues.apache.org/jira/browse/HBASE-15324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li updated HBASE-15324:
--
Status: Patch Available (was: Open)
Submit patch to try HadoopQA
> Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy
> and trigger unexpected split
> --
>
> Key: HBASE-15324
> URL: https://issues.apache.org/jira/browse/HBASE-15324
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.3, 2.0.0
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-15324.patch, HBASE-15324_v2.patch
>
>
> We introduce jitter for region split decision in HBASE-13412, but the
> following line in {{ConstantSizeRegionSplitPolicy}} may cause long value
> overflow if MAX_FILESIZE is specified to Long.MAX_VALUE:
> {code}
> this.desiredMaxFileSize += (long)(desiredMaxFileSize * (RANDOM.nextFloat() -
> 0.5D) * jitter);
> {code}
> In our case we specify MAX_FILESIZE to Long.MAX_VALUE to prevent target
> region to split.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15324) Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy and trigger unexpected split
[
https://issues.apache.org/jira/browse/HBASE-15324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li updated HBASE-15324:
--
Attachment: HBASE-15324_v2.patch
Adding UT for the overflow case, codes w/o patch will cause UT failure.
> Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy
> and trigger unexpected split
> --
>
> Key: HBASE-15324
> URL: https://issues.apache.org/jira/browse/HBASE-15324
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.1.3
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-15324.patch, HBASE-15324_v2.patch
>
>
> We introduce jitter for region split decision in HBASE-13412, but the
> following line in {{ConstantSizeRegionSplitPolicy}} may cause long value
> overflow if MAX_FILESIZE is specified to Long.MAX_VALUE:
> {code}
> this.desiredMaxFileSize += (long)(desiredMaxFileSize * (RANDOM.nextFloat() -
> 0.5D) * jitter);
> {code}
> In our case we specify MAX_FILESIZE to Long.MAX_VALUE to prevent target
> region to split.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15321) Ability to open a HRegion from hdfs snapshot.
[
https://issues.apache.org/jira/browse/HBASE-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166852#comment-15166852
]
Hadoop QA commented on HBASE-15321:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
1s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m 19s
{color} | {color:red} Patch generated 8 new checkstyle issues in hbase-server
(total was 251, now 259). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 34s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 46s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 16m 40s {color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 11s
{color} | {color:red} Patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 80m 27s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_72 Failed junit tests |
hadoop.hbase.regionserver.TestMetricsRegionServer |
| | hadoop.hbase.regionserver.TestMetricsRegion |
| | hadoop.hbase.ipc.TestRpcMetrics |
| JDK v1.7.0_95 Failed junit tests |
hadoop.hbase.regionserver.TestMetricsRegionServer |
| | hadoop.hbase.regionserver.TestMetricsRegion |
| | hadoop.hbase.ipc.TestRpcMetrics |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.9.1 Server=1.9.1 Image:yetus/hbase:date2016-02-25 |
| JIRA Patch URL
[jira] [Commented] (HBASE-15325) ResultScanner allowing partial result will reset to the start of the row if the region is moved between two rpc requests
[
https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166847#comment-15166847
]
Jianwei Cui commented on HBASE-15325:
-
In ScannerCallable#call(), the NotServingRegionException will be wrapped as
DoNotRetryIOException:
{code}
if (ioe instanceof NotServingRegionException) {
// Throw a DNRE so that we break out of cycle of calling NSRE
// when what we need is to open scanner against new location.
// Attach NSRE to signal client that it needs to re-setup scanner.
if (this.scanMetrics != null) {
this.scanMetrics.countOfNSRE.incrementAndGet();
}
throw new DoNotRetryIOException("Resetting the scanner -- see
exception cause", ioe);
{code}
> ResultScanner allowing partial result will reset to the start of the row if
> the region is moved between two rpc requests
>
>
> Key: HBASE-15325
> URL: https://issues.apache.org/jira/browse/HBASE-15325
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.3
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Attachments: 15325-test.txt
>
>
> HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for
> one rpc request. And client can setAllowPartial or setBatch to get several
> cells in a row instead of the whole row.
> However, the status of the scanner is saved on server and we need this to get
> the next part if there is a partial result before. If we move the region to
> another RS, client will get a NotServingRegionException and open a new
> scanner to the new RS which will be regarded as a new scan from the start of
> this row. So we will see the cells which have been seen before.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[
https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166846#comment-15166846
]
Hadoop QA commented on HBASE-6721:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 15 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 35s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 5s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 25m
35s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
4s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 4m
0s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 10m 25s
{color} | {color:red} branch/. no findbugs output file
(./target/findbugsXml.xml) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 48s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 46s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 18s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 32s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 32s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 13m 3s
{color} | {color:red} Patch generated 17 new checkstyle issues in root (total
was 637, now 650). {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 38s
{color} | {color:red} Patch generated 3 new checkstyle issues in hbase-client
(total was 188, now 189). {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 12s
{color} | {color:red} Patch generated 4 new checkstyle issues in hbase-rsgroup
(total was 0, now 4). {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 3m 55s
{color} | {color:red} Patch generated 10 new checkstyle issues in hbase-server
(total was 415, now 424). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
16s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m 28s
{color} | {color:red} The applied patch generated 137 new rubocop issues (total
was 263, now 399). {color} |
| {color:red}-1{color} | {color:red} ruby-lint {color} | {color:red} 0m 11s
{color} | {color:red} The applied patch generated 68 new ruby-lint issues
(total was 137, now 205). {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 1s
{color} | {color:red} The patch has 94 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 2s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 22s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.
[jira] [Commented] (HBASE-14918) In-Memory MemStore Flush and Compaction
[ https://issues.apache.org/jira/browse/HBASE-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166843#comment-15166843 ] Anoop Sam John commented on HBASE-14918: So the underlying data structure is array of Cells or array of PBRs? HBASE-10713, I considered the in memory flushed CellBlock to have a plain byte[] representation as we can see in HFile data blocks. An array's overhead seems not that much compared to as CSLM. So array is ok. Ya it helps with a binary search and things looks much simpler. Regarding PBR return type from MSLAB, this will create issue with off heap MSLAB. So we try to change this to BB. Java ByteBuffer type is enough not our ByteBuff. Our ByteBuff adds one unwanted wrap. We dont want multiple BB backing for return for each of the allocate call to MSLAB, FYI [~saint@gmail.com] > In-Memory MemStore Flush and Compaction > --- > > Key: HBASE-14918 > URL: https://issues.apache.org/jira/browse/HBASE-14918 > Project: HBase > Issue Type: Umbrella >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Fix For: 0.98.18 > > Attachments: CellBlocksSegmentDesign.pdf, MSLABMove.patch > > > A memstore serves as the in-memory component of a store unit, absorbing all > updates to the store. From time to time these updates are flushed to a file > on disk, where they are compacted (by eliminating redundancies) and > compressed (i.e., written in a compressed format to reduce their storage > size). > We aim to speed up data access, and therefore suggest to apply in-memory > memstore flush. That is to flush the active in-memory segment into an > intermediate buffer where it can be accessed by the application. Data in the > buffer is subject to compaction and can be stored in any format that allows > it to take up smaller space in RAM. The less space the buffer consumes the > longer it can reside in memory before data is flushed to disk, resulting in > better performance. > Specifically, the optimization is beneficial for workloads with > medium-to-high key churn which incur many redundant cells, like persistent > messaging. > We suggest to structure the solution as 4 subtasks (respectively, patches). > (1) Infrastructure - refactoring of the MemStore hierarchy, introducing > segment (StoreSegment) as first-class citizen, and decoupling memstore > scanner from the memstore implementation; > (2) Adding StoreServices facility at the region level to allow memstores > update region counters and access region level synchronization mechanism; > (3) Implementation of a new memstore (CompactingMemstore) with non-optimized > immutable segment representation, and > (4) Memory optimization including compressed format representation and off > heap allocations. > This Jira continues the discussion in HBASE-13408. > Design documents, evaluation results and previous patches can be found in > HBASE-13408. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166841#comment-15166841 ] stack commented on HBASE-15205: --- I'd be +1 on this. Looks much better. We can clean up clearing of scopes in certain contexts in another patch. Seems to be too many constructors on WALKey but can fix that in another patch too. Thanks for putting up w/ my review [~ram_krish] > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, HBASE-15205_8.patch, HBASE-15205_9.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166838#comment-15166838 ] Hudson commented on HBASE-15136: SUCCESS: Integrated in HBase-1.3-IT #516 (See [https://builds.apache.org/job/HBase-1.3-IT/516/]) HBASE-15136 Explore different queuing behaviors while busy (antonov: rev 04a3b27330839a33d75f71ac2d9997abc3c83957) * hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestSimpleRpcScheduler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RWQueueRpcExecutor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerWrapperStub.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/FifoRpcScheduler.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSource.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcScheduler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerWrapperImpl.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerWrapper.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/AdaptiveLifoCoDelCallQueue.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/ReflectionUtils.java > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15016) StoreServices facility in Region
[ https://issues.apache.org/jira/browse/HBASE-15016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166831#comment-15166831 ] Eshcar Hillel commented on HBASE-15016: --- Excellent!!! Moving on to work on Task #3 > StoreServices facility in Region > > > Key: HBASE-15016 > URL: https://issues.apache.org/jira/browse/HBASE-15016 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Fix For: 2.0.0 > > Attachments: HBASE-15016-V01.patch, HBASE-15016-V02.patch, > HBASE-15016-V03 (1).patch, HBASE-15016-V03.patch, HBASE-15016-V04.patch, > HBASE-15016-V05.patch, HBASE-15016-V05.patch, HBASE-15016-V06.patch, > Regioncounters.pdf, suggestion.patch > > > The default implementation of a memstore ensures that between two flushes the > memstore size increases monotonically. Supporting new memstores that store > data in different formats (specifically, compressed), or that allows to > eliminate data redundancies in memory (e.g., via compaction), means that the > size of the data stored in memory can decrease even between two flushes. This > requires memstores to have access to facilities that manipulate region > counters and synchronization. > This subtasks introduces a new region interface -- StoreServices, through > which store components can access these facilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15222) Use less contended classes for metrics
[
https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166829#comment-15166829
]
Duo Zhang commented on HBASE-15222:
---
For {{TestRpcMetrics}}, first we need to add null check for two method. One is
{{MutableHistogram.snapshot}} and the other is
{{MutableRangeHistogram.snapshot}}.
If the returned {{FastLongHistogram}} is null then we should not call
updateSnapshotMetrics.
The null check can fix some of the problems. And there is still another problem
which seems some of our metrics can only be retrieved once.
{code:title=TestRpcMetrics.java}
mrpc.dequeuedCall(100);
mrpc.processedCall(101);
mrpc.totalCall(102);
HELPER.assertCounter("queueCallTime_NumOps", 1, serverSource);
HELPER.assertCounter("processCallTime_NumOps", 1, serverSource);
HELPER.assertCounter("totalCallTime_NumOps", 1, serverSource);
{code}
{code:title=TestRpcMetrics.java}
mrpc.dequeuedCall(100);
HELPER.assertCounter("queueCallTime_NumOps", 1, serverSource);
mrpc.processedCall(101);
HELPER.assertCounter("processCallTime_NumOps", 1, serverSource);
mrpc.totalCall(102);
HELPER.assertCounter("totalCallTime_NumOps", 1, serverSource);
{code}
The former one will fail but the latter one will succeed. Is this what we
expected? Thanks.
> Use less contended classes for metrics
> --
>
> Key: HBASE-15222
> URL: https://issues.apache.org/jira/browse/HBASE-15222
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.0.0, 1.2.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Critical
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-15222-v1.patch, HBASE-15222-v10.patch,
> HBASE-15222-v11.patch, HBASE-15222-v12.patch, HBASE-15222-v13.patch,
> HBASE-15222-v2.patch, HBASE-15222-v3.patch, HBASE-15222-v5.patch,
> HBASE-15222-v6.patch, HBASE-15222-v8.patch, HBASE-15222-v9.patch,
> HBASE-15222.patch
>
>
> Running the benchmarks now, but it looks like the results are pretty extreme.
> The locking in our histograms is pretty extreme.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Patch Available (was: Open) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, HBASE-15205_8.patch, HBASE-15205_9.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Attachment: HBASE-15205_9.patch Rebased patch for QA. > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, HBASE-15205_8.patch, HBASE-15205_9.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Open (was: Patch Available) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, HBASE-15205_8.patch, ScopeWALEdits.jpg, > ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15325) ResultScanner allowing partial result will reset to the start of the row if the region is moved between two rpc requests
[ https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166821#comment-15166821 ] Phil Yang commented on HBASE-15325: --- NotServingRegionException extends IOException so it is not DoNotRetryIOException. And I am not sure if this exception will be thrown to ClientScanner, may be handled in a lower level? > ResultScanner allowing partial result will reset to the start of the row if > the region is moved between two rpc requests > > > Key: HBASE-15325 > URL: https://issues.apache.org/jira/browse/HBASE-15325 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Phil Yang >Assignee: Phil Yang >Priority: Critical > Attachments: 15325-test.txt > > > HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for > one rpc request. And client can setAllowPartial or setBatch to get several > cells in a row instead of the whole row. > However, the status of the scanner is saved on server and we need this to get > the next part if there is a partial result before. If we move the region to > another RS, client will get a NotServingRegionException and open a new > scanner to the new RS which will be regarded as a new scan from the start of > this row. So we will see the cells which have been seen before. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15325) ResultScanner allowing partial result will reset to the start of the row if the region is moved between two rpc requests
[
https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166818#comment-15166818
]
Anoop Sam John commented on HBASE-15325:
I see this in code but it do not consider the partial Results the user already
pulled out of the cache
{code}
} catch (DoNotRetryIOException e) {
// An exception was thrown which makes any partial results that we were
collecting
// invalid. The scanner will need to be reset to the beginning of a row.
clearPartialResults();
{code}
So said that, we will get again the same cells which would have already
fetched. Ya this is more work for server but seems a simple solution
To say to the server to start scan not at begin of a row looks to be more
complex change.
> ResultScanner allowing partial result will reset to the start of the row if
> the region is moved between two rpc requests
>
>
> Key: HBASE-15325
> URL: https://issues.apache.org/jira/browse/HBASE-15325
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.3
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Attachments: 15325-test.txt
>
>
> HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for
> one rpc request. And client can setAllowPartial or setBatch to get several
> cells in a row instead of the whole row.
> However, the status of the scanner is saved on server and we need this to get
> the next part if there is a partial result before. If we move the region to
> another RS, client will get a NotServingRegionException and open a new
> scanner to the new RS which will be regarded as a new scan from the start of
> this row. So we will see the cells which have been seen before.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15324) Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy and trigger unexpected split
[
https://issues.apache.org/jira/browse/HBASE-15324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166812#comment-15166812
]
Yu Li commented on HBASE-15324:
---
Another thing I'd like to mention is that currently there's no attribute in
HTableDescriptor to mark a table not splittable, and we have to achieve this by
specifying MAX_FILESIZE to Long.MAX_VALUE. I think we could introduce a new
attribute like NEVER_SPLIT to make it much easier and straight-forward.
Thoughts on this idea? If agreed, I could open another JIRA to implement it.
Thanks.
> Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy
> and trigger unexpected split
> --
>
> Key: HBASE-15324
> URL: https://issues.apache.org/jira/browse/HBASE-15324
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.1.3
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-15324.patch
>
>
> We introduce jitter for region split decision in HBASE-13412, but the
> following line in {{ConstantSizeRegionSplitPolicy}} may cause long value
> overflow if MAX_FILESIZE is specified to Long.MAX_VALUE:
> {code}
> this.desiredMaxFileSize += (long)(desiredMaxFileSize * (RANDOM.nextFloat() -
> 0.5D) * jitter);
> {code}
> In our case we specify MAX_FILESIZE to Long.MAX_VALUE to prevent target
> region to split.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15325) ResultScanner allowing partial result will reset to the start of the row if the region is moved between two rpc requests
[ https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166811#comment-15166811 ] Phil Yang commented on HBASE-15325: --- I think we have two solutions: Cache the last cell of the last result from loadcache(), and skip all Cells not larger than this when we loadcache() next time. It is simple but will need server response some repeat cells. Cache the last cell of the last result from loadcache(), and tell the new server where we should start for this row. Server can seek to the right position and response the right cells. It seems that seek is also a "skip" logic so the second solution will use the same MemStore/Disk resource, so the only difference is network and time usage. Feel free to correct me if I am wrong. > ResultScanner allowing partial result will reset to the start of the row if > the region is moved between two rpc requests > > > Key: HBASE-15325 > URL: https://issues.apache.org/jira/browse/HBASE-15325 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Phil Yang >Assignee: Phil Yang >Priority: Critical > Attachments: 15325-test.txt > > > HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for > one rpc request. And client can setAllowPartial or setBatch to get several > cells in a row instead of the whole row. > However, the status of the scanner is saved on server and we need this to get > the next part if there is a partial result before. If we move the region to > another RS, client will get a NotServingRegionException and open a new > scanner to the new RS which will be regarded as a new scan from the start of > this row. So we will see the cells which have been seen before. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15322) HBase 1.1.3 crashing
[ https://issues.apache.org/jira/browse/HBASE-15322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166805#comment-15166805 ] Anoop Sam John commented on HBASE-15322: I mean com.sun.Unsafe class > HBase 1.1.3 crashing > > > Key: HBASE-15322 > URL: https://issues.apache.org/jira/browse/HBASE-15322 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.1.3 > Environment: OS: Ubuntu 14.04/Ubuntu 15.10 > JDK: OpenJDK8/OpenJDK9 >Reporter: Anant Sharma > > HBase crashes in standalone mode with the following log: > __ > 2016-02-24 22:38:37,578 ERROR [main] master.HMasterCommandLine: Master exiting > java.lang.RuntimeException: Failed construction of Master: class > org.apache.hadoop.hbase.master.HMaster > at > org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2341) > at > org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:233) > at > org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) > at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2355) > Caused by: java.lang.NoClassDefFoundError: Could not initialize class > org.apache.hadoop.hbase.util.Bytes$LexicographicalComparerHolder$UnsafeComparer > at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:899) > at > org.apache.hadoop.hbase.KeyValue.createByteArray(KeyValue.java:1082) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:652) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:580) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:483) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:370) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:267) > at org.apache.hadoop.hbase.HConstants.(HConstants.java:978) > at > org.apache.hadoop.hbase.HTableDescriptor.(HTableDescriptor.java:1488) > at > org.apache.hadoop.hbase.util.FSTableDescriptors.(FSTableDescriptors.java:124) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:570) > at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:365) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2336) > __ > The class is in the hbase-common.jar and its there in the classpath as can be > seen from the log: > _ > 2016-02-24 22:38:32,538 INFO [main] util.ServerCommandLine: > env:CLASSPATH=/home/hduser/hbase/hbase-1.1.3:/home/hduser/hbase/hbase-1.1.3/lib/activation-1.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/aopalliance-1.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/apacheds-i18n-2.0.0-M15.jar:/home/hduser/hbase/hbase-1.1.3/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/hduser/hbase/hbase-1.1.3/lib/api-asn1-api-1.0.0-M20.jar:/home/hduser/hbase/hbase-1.1.3/lib/api-util-1.0.0-M20.jar:/home/hduser/hbase/hbase-1.1.3/lib/asm-3.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/avro-1.7.4.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-beanutils-1.7.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-beanutils-core-1.8.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-cli-1.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-codec-1.9.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-collections-3.2.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-compress-1.4.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-configuration-1.6.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-daemon-1.0.13.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-digester-1.8.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-el-1.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-httpclient-3.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-io-2.4.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-lang-2.6.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-logging-1.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-math-2.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-math3-3.1.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-net-3.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/disruptor-3.3.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/findbugs-annotations-1.3.9-1.jar:/home/hdu
[jira] [Commented] (HBASE-15205) Do not find the replication scope for every WAL#append()
[
https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166802#comment-15166802
]
Hadoop QA commented on HBASE-15205:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HBASE-15205 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/latest/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12789877/HBASE-15205_8.patch |
| JIRA Issue | HBASE-15205 |
| Powered by | Apache Yetus 0.1.0 http://yetus.apache.org |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/693/console |
This message was automatically generated.
> Do not find the replication scope for every WAL#append()
>
>
> Key: HBASE-15205
> URL: https://issues.apache.org/jira/browse/HBASE-15205
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15204_6.patch, HBASE-15205.patch,
> HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch,
> HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch,
> HBASE-15205_7.patch, HBASE-15205_8.patch, ScopeWALEdits.jpg,
> ScopeWALEdits_afterpatch.jpg
>
>
> After the byte[] and char[] the other top contributor for lot of GC (though
> it is only 2.86%) is the UTF_8.newDecoder.
> This happens because for every WAL append we try to calculate the replication
> scope associate with the families associated with the TableDescriptor. I
> think per WAL append doing this is very costly and creates lot of garbage.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15322) HBase 1.1.3 crashing
[ https://issues.apache.org/jira/browse/HBASE-15322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166801#comment-15166801 ] Anant Sharma commented on HBASE-15322: -- Its there with hadoop-common.jar Even after adding this class in the classpath using HBASE_CLASSPATH env variable I get the same error > HBase 1.1.3 crashing > > > Key: HBASE-15322 > URL: https://issues.apache.org/jira/browse/HBASE-15322 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.1.3 > Environment: OS: Ubuntu 14.04/Ubuntu 15.10 > JDK: OpenJDK8/OpenJDK9 >Reporter: Anant Sharma > > HBase crashes in standalone mode with the following log: > __ > 2016-02-24 22:38:37,578 ERROR [main] master.HMasterCommandLine: Master exiting > java.lang.RuntimeException: Failed construction of Master: class > org.apache.hadoop.hbase.master.HMaster > at > org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2341) > at > org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:233) > at > org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) > at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2355) > Caused by: java.lang.NoClassDefFoundError: Could not initialize class > org.apache.hadoop.hbase.util.Bytes$LexicographicalComparerHolder$UnsafeComparer > at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:899) > at > org.apache.hadoop.hbase.KeyValue.createByteArray(KeyValue.java:1082) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:652) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:580) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:483) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:370) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:267) > at org.apache.hadoop.hbase.HConstants.(HConstants.java:978) > at > org.apache.hadoop.hbase.HTableDescriptor.(HTableDescriptor.java:1488) > at > org.apache.hadoop.hbase.util.FSTableDescriptors.(FSTableDescriptors.java:124) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:570) > at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:365) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2336) > __ > The class is in the hbase-common.jar and its there in the classpath as can be > seen from the log: > _ > 2016-02-24 22:38:32,538 INFO [main] util.ServerCommandLine: > env:CLASSPATH=/home/hduser/hbase/hbase-1.1.3:/home/hduser/hbase/hbase-1.1.3/lib/activation-1.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/aopalliance-1.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/apacheds-i18n-2.0.0-M15.jar:/home/hduser/hbase/hbase-1.1.3/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/hduser/hbase/hbase-1.1.3/lib/api-asn1-api-1.0.0-M20.jar:/home/hduser/hbase/hbase-1.1.3/lib/api-util-1.0.0-M20.jar:/home/hduser/hbase/hbase-1.1.3/lib/asm-3.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/avro-1.7.4.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-beanutils-1.7.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-beanutils-core-1.8.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-cli-1.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-codec-1.9.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-collections-3.2.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-compress-1.4.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-configuration-1.6.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-daemon-1.0.13.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-digester-1.8.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-el-1.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-httpclient-3.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-io-2.4.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-lang-2.6.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-logging-1.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-math-2.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-math3-3.1.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-net-3.1.jar:/home/hduser/hbase/hbase-1.1
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[
https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-15205:
---
Attachment: HBASE-15205_8.patch
Latest patch. Removed replicationScope from FSWALEntry. WALKey construction
itself will now have replicationScope. As I said earlier since we are now
passing the scope during WALKey construction itself now it is upto us to
nullify the scopes in logKey when ever it is not needed. I felt that was not
looking clean. Still we are removing HTD from WAL#append() and things work as
in previous patches.
Now we will call this
{code}
+if ((!replicationForBulkLoadEnabled && !foundOtherEdits) ||
logEdit.isReplay()) {
+ logKey.serializeReplicationScope(false);
}
{code}
instead of setReplciationScope(). Inside WALKEy#serializeReplicationScope() we
will just nullify the replicationScope that was passed during construction.
> Do not find the replication scope for every WAL#append()
>
>
> Key: HBASE-15205
> URL: https://issues.apache.org/jira/browse/HBASE-15205
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15204_6.patch, HBASE-15205.patch,
> HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch,
> HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch,
> HBASE-15205_7.patch, HBASE-15205_8.patch, ScopeWALEdits.jpg,
> ScopeWALEdits_afterpatch.jpg
>
>
> After the byte[] and char[] the other top contributor for lot of GC (though
> it is only 2.86%) is the UTF_8.newDecoder.
> This happens because for every WAL append we try to calculate the replication
> scope associate with the families associated with the TableDescriptor. I
> think per WAL append doing this is very costly and creates lot of garbage.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Patch Available (was: Open) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, HBASE-15205_8.patch, ScopeWALEdits.jpg, > ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15324) Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy and trigger unexpected split
[
https://issues.apache.org/jira/browse/HBASE-15324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li updated HBASE-15324:
--
Attachment: HBASE-15324.patch
A straight forward patch to fix the issue.
To supplement, the region will split with small store size when overflow occurs
and cause really bad perf issue.
> Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy
> and trigger unexpected split
> --
>
> Key: HBASE-15324
> URL: https://issues.apache.org/jira/browse/HBASE-15324
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.1.3
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-15324.patch
>
>
> We introduce jitter for region split decision in HBASE-13412, but the
> following line in {{ConstantSizeRegionSplitPolicy}} may cause long value
> overflow if MAX_FILESIZE is specified to Long.MAX_VALUE:
> {code}
> this.desiredMaxFileSize += (long)(desiredMaxFileSize * (RANDOM.nextFloat() -
> 0.5D) * jitter);
> {code}
> In our case we specify MAX_FILESIZE to Long.MAX_VALUE to prevent target
> region to split.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15325) ResultScanner allowing partial result will reset to the start of the row if the region is moved between two rpc requests
[ https://issues.apache.org/jira/browse/HBASE-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-15325: -- Attachment: 15325-test.txt Upload two tests for this issue. The testPartialResultWhenRegionMove will break on line 887 while testCompleteResultWhenRegionMove will pass. But I am not sure what will happen if we move a region between two next() when we setAllowPartialResults(false), i am digging into it. > ResultScanner allowing partial result will reset to the start of the row if > the region is moved between two rpc requests > > > Key: HBASE-15325 > URL: https://issues.apache.org/jira/browse/HBASE-15325 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3 >Reporter: Phil Yang >Assignee: Phil Yang >Priority: Critical > Attachments: 15325-test.txt > > > HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for > one rpc request. And client can setAllowPartial or setBatch to get several > cells in a row instead of the whole row. > However, the status of the scanner is saved on server and we need this to get > the next part if there is a partial result before. If we move the region to > another RS, client will get a NotServingRegionException and open a new > scanner to the new RS which will be regarded as a new scan from the start of > this row. So we will see the cells which have been seen before. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Open (was: Patch Available) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15319) clearJmxCache does not take effect actually
[
https://issues.apache.org/jira/browse/HBASE-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166794#comment-15166794
]
Hudson commented on HBASE-15319:
SUCCESS: Integrated in HBase-1.2 #562 (See
[https://builds.apache.org/job/HBase-1.2/562/])
HBASE-15319 clearJmxCache does not take effect actually (eclark: rev
4710463d2d7191425cc54ebe4d62e3f4ea6672dd)
*
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/impl/JmxCacheBuster.java
> clearJmxCache does not take effect actually
> ---
>
> Key: HBASE-15319
> URL: https://issues.apache.org/jira/browse/HBASE-15319
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.0.0, 1.2.0, 1.1.0.1, 1.3.0
>Reporter: Hao Lin
>Assignee: Elliott Clark
> Labels: easyfix
> Fix For: 2.0.0, 1.3.0, 1.2.1
>
> Attachments: HBASE-15319.patch
>
>
> When trying to backport HBASE-14166 to 0.98.6, I find
> JmxCacheBuster::clearJmxCache() does no take effect actually. The related
> code are listed below:
> {code:title=org.apache.hadoop.metrics2.impl.JmxCacheBuster.java|borderStyle=solid}
> // fut is initialized to null
> private static AtomicReference fut = new
> AtomicReference<>(null);
> public static void clearJmxCache() {
> // clearJmxCache return directly when fut is null, which is always true.
> // the actual intent is 'if (future != null && !future.isDone ...)' ?
> ScheduledFuture future = fut.get();
> if ((future == null || (!future.isDone() &&
> future.getDelay(TimeUnit.MILLISECONDS) > 100))) {
> return;
> }
> ..
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15325) ResultScanner allowing partial result will reset to the start of the row if the region is moved between two rpc requests
Phil Yang created HBASE-15325: - Summary: ResultScanner allowing partial result will reset to the start of the row if the region is moved between two rpc requests Key: HBASE-15325 URL: https://issues.apache.org/jira/browse/HBASE-15325 Project: HBase Issue Type: Bug Affects Versions: 1.1.3 Reporter: Phil Yang Assignee: Phil Yang Priority: Critical HBASE-11544 allow scan rpc return partial of a row to reduce memory usage for one rpc request. And client can setAllowPartial or setBatch to get several cells in a row instead of the whole row. However, the status of the scanner is saved on server and we need this to get the next part if there is a partial result before. If we move the region to another RS, client will get a NotServingRegionException and open a new scanner to the new RS which will be regarded as a new scan from the start of this row. So we will see the cells which have been seen before. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15324) Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy and trigger unexpected split
Yu Li created HBASE-15324:
-
Summary: Jitter may cause desiredMaxFileSize overflow in
ConstantSizeRegionSplitPolicy and trigger unexpected split
Key: HBASE-15324
URL: https://issues.apache.org/jira/browse/HBASE-15324
Project: HBase
Issue Type: Bug
Affects Versions: 1.1.3, 2.0.0
Reporter: Yu Li
Assignee: Yu Li
We introduce jitter for region split decision in HBASE-13412, but the following
line in {{ConstantSizeRegionSplitPolicy}} may cause long value overflow if
MAX_FILESIZE is specified to Long.MAX_VALUE:
{code}
this.desiredMaxFileSize += (long)(desiredMaxFileSize * (RANDOM.nextFloat() -
0.5D) * jitter);
{code}
In our case we specify MAX_FILESIZE to Long.MAX_VALUE to prevent target region
to split.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15319) clearJmxCache does not take effect actually
[
https://issues.apache.org/jira/browse/HBASE-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166783#comment-15166783
]
Hudson commented on HBASE-15319:
FAILURE: Integrated in HBase-1.3 #570 (See
[https://builds.apache.org/job/HBase-1.3/570/])
HBASE-15319 clearJmxCache does not take effect actually (eclark: rev
13773254c87994c6cffdb3fbc48521d51c86d8c5)
*
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/impl/JmxCacheBuster.java
> clearJmxCache does not take effect actually
> ---
>
> Key: HBASE-15319
> URL: https://issues.apache.org/jira/browse/HBASE-15319
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.0.0, 1.2.0, 1.1.0.1, 1.3.0
>Reporter: Hao Lin
>Assignee: Elliott Clark
> Labels: easyfix
> Fix For: 2.0.0, 1.3.0, 1.2.1
>
> Attachments: HBASE-15319.patch
>
>
> When trying to backport HBASE-14166 to 0.98.6, I find
> JmxCacheBuster::clearJmxCache() does no take effect actually. The related
> code are listed below:
> {code:title=org.apache.hadoop.metrics2.impl.JmxCacheBuster.java|borderStyle=solid}
> // fut is initialized to null
> private static AtomicReference fut = new
> AtomicReference<>(null);
> public static void clearJmxCache() {
> // clearJmxCache return directly when fut is null, which is always true.
> // the actual intent is 'if (future != null && !future.isDone ...)' ?
> ScheduledFuture future = fut.get();
> if ((future == null || (!future.isDone() &&
> future.getDelay(TimeUnit.MILLISECONDS) > 100))) {
> return;
> }
> ..
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15322) HBase 1.1.3 crashing
[ https://issues.apache.org/jira/browse/HBASE-15322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166746#comment-15166746 ] Anoop Sam John commented on HBASE-15322: Unsafe class is not available in your classpath? Can u check > HBase 1.1.3 crashing > > > Key: HBASE-15322 > URL: https://issues.apache.org/jira/browse/HBASE-15322 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.1.3 > Environment: OS: Ubuntu 14.04/Ubuntu 15.10 > JDK: OpenJDK8/OpenJDK9 >Reporter: Anant Sharma > > HBase crashes in standalone mode with the following log: > __ > 2016-02-24 22:38:37,578 ERROR [main] master.HMasterCommandLine: Master exiting > java.lang.RuntimeException: Failed construction of Master: class > org.apache.hadoop.hbase.master.HMaster > at > org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2341) > at > org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:233) > at > org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) > at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2355) > Caused by: java.lang.NoClassDefFoundError: Could not initialize class > org.apache.hadoop.hbase.util.Bytes$LexicographicalComparerHolder$UnsafeComparer > at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:899) > at > org.apache.hadoop.hbase.KeyValue.createByteArray(KeyValue.java:1082) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:652) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:580) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:483) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:370) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:267) > at org.apache.hadoop.hbase.HConstants.(HConstants.java:978) > at > org.apache.hadoop.hbase.HTableDescriptor.(HTableDescriptor.java:1488) > at > org.apache.hadoop.hbase.util.FSTableDescriptors.(FSTableDescriptors.java:124) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:570) > at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:365) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2336) > __ > The class is in the hbase-common.jar and its there in the classpath as can be > seen from the log: > _ > 2016-02-24 22:38:32,538 INFO [main] util.ServerCommandLine: > env:CLASSPATH=/home/hduser/hbase/hbase-1.1.3:/home/hduser/hbase/hbase-1.1.3/lib/activation-1.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/aopalliance-1.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/apacheds-i18n-2.0.0-M15.jar:/home/hduser/hbase/hbase-1.1.3/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/hduser/hbase/hbase-1.1.3/lib/api-asn1-api-1.0.0-M20.jar:/home/hduser/hbase/hbase-1.1.3/lib/api-util-1.0.0-M20.jar:/home/hduser/hbase/hbase-1.1.3/lib/asm-3.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/avro-1.7.4.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-beanutils-1.7.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-beanutils-core-1.8.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-cli-1.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-codec-1.9.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-collections-3.2.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-compress-1.4.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-configuration-1.6.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-daemon-1.0.13.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-digester-1.8.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-el-1.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-httpclient-3.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-io-2.4.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-lang-2.6.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-logging-1.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-math-2.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-math3-3.1.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-net-3.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/disruptor-3.3.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/findbugs-
[jira] [Commented] (HBASE-15222) Use less contended classes for metrics
[
https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166747#comment-15166747
]
Duo Zhang commented on HBASE-15222:
---
Seems this causes several tests fail?
See
https://builds.apache.org/job/PreCommit-HBASE-Build/690
https://builds.apache.org/job/PreCommit-HBASE-Build/691
I tried locally, {{TestRpcMetrics.testSourceMethods()}} failed with NPE.
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.metrics2.lib.MutableHistogram.updateSnapshotMetrics(MutableHistogram.java:72)
at
org.apache.hadoop.metrics2.lib.MutableRangeHistogram.snapshot(MutableRangeHistogram.java:59)
at
org.apache.hadoop.metrics2.lib.DynamicMetricsRegistry.snapshot(DynamicMetricsRegistry.java:391)
at
org.apache.hadoop.hbase.ipc.MetricsHBaseServerSourceImpl.getMetrics(MetricsHBaseServerSourceImpl.java:225)
at
org.apache.hadoop.hbase.test.MetricsAssertHelperImpl.getMetrics(MetricsAssertHelperImpl.java:243)
at
org.apache.hadoop.hbase.test.MetricsAssertHelperImpl.getCounter(MetricsAssertHelperImpl.java:201)
at
org.apache.hadoop.hbase.test.MetricsAssertHelperImpl.assertCounter(MetricsAssertHelperImpl.java:183)
at
org.apache.hadoop.hbase.ipc.TestRpcMetrics.testSourceMethods(TestRpcMetrics.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
{noformat}
> Use less contended classes for metrics
> --
>
> Key: HBASE-15222
> URL: https://issues.apache.org/jira/browse/HBASE-15222
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.0.0, 1.2.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Critical
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-15222-v1.patch, HBASE-15222-v10.patch,
> HBASE-15222-v11.patch, HBASE-15222-v12.patch, HBASE-15222-v13.patch,
> HBASE-15222-v2.patch, HBASE-15222-v3.patch, HBASE-15222-v5.patch,
> HBASE-15222-v6.patch, HBASE-15222-v8.patch, HBASE-15222-v9.patch,
> HBASE-15222.patch
>
>
> Running the benchmarks now, but it looks like the results are pretty extreme.
> The locking in our histograms is pretty extreme.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15265) Implement an asynchronous FSHLog
[
https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166739#comment-15166739
]
Hadoop QA commented on HBASE-15265:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
1s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 14 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
7s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
9s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m 13s
{color} | {color:red} Patch generated 23 new checkstyle issues in hbase-server
(total was 84, now 84). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 37s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 4s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 3m 0s
{color} | {color:red} hbase-server-jdk1.8.0_72 with JDK v1.8.0_72 generated 3
new issues (was 1, now 4). {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 3m 55s
{color} | {color:red} hbase-server-jdk1.7.0_95 with JDK v1.7.0_95 generated 3
new issues (was 1, now 4). {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 43s
{color} | {color:green} hbase-common in the patch passed with JDK v1.8.0_72.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 16m 1s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 49s
{color} | {color:green} hbase-common in the patch passed with JDK v1.7.0_95.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 17m 50s {color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_95. {color} |
| {color:green}+1{color}
[jira] [Commented] (HBASE-15302) Reenable the other tests disabled by HBASE-14678
[
https://issues.apache.org/jira/browse/HBASE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166731#comment-15166731
]
Hadoop QA commented on HBASE-15302:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 30s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/latest/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 8 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 24s
{color} | {color:green} branch-1 passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 10s
{color} | {color:green} branch-1 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
54s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
50s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 12s
{color} | {color:green} branch-1 passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s
{color} | {color:green} branch-1 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 27s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 27s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 7s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 6m
38s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 25m 43s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 29s
{color} | {color:green} hbase-shell in the patch passed with JDK v1.8.0_72.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 23m 10s {color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 25s
{color} | {color:green} hbase-shell in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 82m 19s {color}
| {col
[jira] [Updated] (HBASE-15321) Ability to open a HRegion from hdfs snapshot.
[ https://issues.apache.org/jira/browse/HBASE-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15321: --- Status: Patch Available (was: Open) > Ability to open a HRegion from hdfs snapshot. > - > > Key: HBASE-15321 > URL: https://issues.apache.org/jira/browse/HBASE-15321 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: churro morales > Fix For: 2.0.0 > > Attachments: HBASE-15321-v1.patch, HBASE-15321.patch > > > Now that hdfs snapshots are here, we started to run our mapreduce jobs over > hdfs snapshots. The thing is, hdfs snapshots are read-only point-in-time > copies of the file system. Thus we had to modify the section of code that > initialized the region internals in HRegion. We have to skip cleanup of > certain directories if the HRegion is backed by a hdfs snapshot. I have a > patch for trunk with some basic tests if folks are interested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15321) Ability to open a HRegion from hdfs snapshot.
[ https://issues.apache.org/jira/browse/HBASE-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-15321: --- Attachment: HBASE-15321-v1.patch fixed the spelling error and closing the snapshotted regions. > Ability to open a HRegion from hdfs snapshot. > - > > Key: HBASE-15321 > URL: https://issues.apache.org/jira/browse/HBASE-15321 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: churro morales > Fix For: 2.0.0 > > Attachments: HBASE-15321-v1.patch, HBASE-15321.patch > > > Now that hdfs snapshots are here, we started to run our mapreduce jobs over > hdfs snapshots. The thing is, hdfs snapshots are read-only point-in-time > copies of the file system. Thus we had to modify the section of code that > initialized the region internals in HRegion. We have to skip cleanup of > certain directories if the HRegion is backed by a hdfs snapshot. I have a > patch for trunk with some basic tests if folks are interested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15136: Resolution: Fixed Status: Resolved (was: Patch Available) > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166726#comment-15166726 ] Mikhail Antonov commented on HBASE-15136: - Thanks - made this change and committed in master and branch-1. > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15321) Ability to open a HRegion from hdfs snapshot.
[
https://issues.apache.org/jira/browse/HBASE-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166725#comment-15166725
]
Ted Yu commented on HBASE-15321:
{code}
60 public void testOpeningReadOnlyRegionBsic() throws Exception {
{code}
Bsic: did you mean Basic ?
Please close the HRegion (in test) which is opened from hdfs snapshot
> Ability to open a HRegion from hdfs snapshot.
> -
>
> Key: HBASE-15321
> URL: https://issues.apache.org/jira/browse/HBASE-15321
> Project: HBase
> Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: churro morales
> Fix For: 2.0.0
>
> Attachments: HBASE-15321.patch
>
>
> Now that hdfs snapshots are here, we started to run our mapreduce jobs over
> hdfs snapshots. The thing is, hdfs snapshots are read-only point-in-time
> copies of the file system. Thus we had to modify the section of code that
> initialized the region internals in HRegion. We have to skip cleanup of
> certain directories if the HRegion is backed by a hdfs snapshot. I have a
> patch for trunk with some basic tests if folks are interested.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15295) MutateTableAccess.multiMutate() does not get high priority causing a deadlock
[
https://issues.apache.org/jira/browse/HBASE-15295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166715#comment-15166715
]
Hadoop QA commented on HBASE-15295:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 16 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 6s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 43s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 8m
4s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
6s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 42s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 40s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 2s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 45s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 45s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 27s
{color} | {color:red} Patch generated 64 new checkstyle issues in hbase-client
(total was 532, now 594). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
6s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
36m 44s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 17s
{color} | {color:red} hbase-server introduced 1 new FindBugs issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 40s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 41s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 27s {color}
| {color:red} hbase-client in the patch failed with JDK v1.8.0_72. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 29s
{color} | {color:green} hbase-it in the patch passed with JDK v1.8.0_72.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 26m 8s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 19s {color}
| {color:red} hbase-client in the patch failed with JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 28s
{color} | {color:green} hbase-it in the patch passed with JDK v1.7.0_95.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 23m 55s {color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s {color} | {color:green} Patch does n
[jira] [Commented] (HBASE-15321) Ability to open a HRegion from hdfs snapshot.
[ https://issues.apache.org/jira/browse/HBASE-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166712#comment-15166712 ] churro morales commented on HBASE-15321: The same way as in TestHdfsSnapshotHRegionit takes an actual hdfs snapshot and goes through the steps of opening a region that was snapshotted. > Ability to open a HRegion from hdfs snapshot. > - > > Key: HBASE-15321 > URL: https://issues.apache.org/jira/browse/HBASE-15321 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: churro morales > Fix For: 2.0.0 > > Attachments: HBASE-15321.patch > > > Now that hdfs snapshots are here, we started to run our mapreduce jobs over > hdfs snapshots. The thing is, hdfs snapshots are read-only point-in-time > copies of the file system. Thus we had to modify the section of code that > initialized the region internals in HRegion. We have to skip cleanup of > certain directories if the HRegion is backed by a hdfs snapshot. I have a > patch for trunk with some basic tests if folks are interested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15321) Ability to open a HRegion from hdfs snapshot.
[
https://issues.apache.org/jira/browse/HBASE-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166708#comment-15166708
]
Ted Yu commented on HBASE-15321:
{code}
6353 public static HRegion openReadOnlyFileSystemHRegion(final
Configuration conf, final FileSystem fs,
6354 final Path rootDir, final Path tableDir, final HRegionInfo info,
final HTableDescriptor htd, final WAL wal)
{code}
How would the above method be called ?
Show example, please.
> Ability to open a HRegion from hdfs snapshot.
> -
>
> Key: HBASE-15321
> URL: https://issues.apache.org/jira/browse/HBASE-15321
> Project: HBase
> Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: churro morales
> Fix For: 2.0.0
>
> Attachments: HBASE-15321.patch
>
>
> Now that hdfs snapshots are here, we started to run our mapreduce jobs over
> hdfs snapshots. The thing is, hdfs snapshots are read-only point-in-time
> copies of the file system. Thus we had to modify the section of code that
> initialized the region internals in HRegion. We have to skip cleanup of
> certain directories if the HRegion is backed by a hdfs snapshot. I have a
> patch for trunk with some basic tests if folks are interested.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15319) clearJmxCache does not take effect actually
[
https://issues.apache.org/jira/browse/HBASE-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166705#comment-15166705
]
Hudson commented on HBASE-15319:
SUCCESS: Integrated in HBase-1.2-IT #447 (See
[https://builds.apache.org/job/HBase-1.2-IT/447/])
HBASE-15319 clearJmxCache does not take effect actually (eclark: rev
4710463d2d7191425cc54ebe4d62e3f4ea6672dd)
*
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/impl/JmxCacheBuster.java
> clearJmxCache does not take effect actually
> ---
>
> Key: HBASE-15319
> URL: https://issues.apache.org/jira/browse/HBASE-15319
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.0.0, 1.2.0, 1.1.0.1, 1.3.0
>Reporter: Hao Lin
>Assignee: Elliott Clark
> Labels: easyfix
> Fix For: 2.0.0, 1.3.0, 1.2.1
>
> Attachments: HBASE-15319.patch
>
>
> When trying to backport HBASE-14166 to 0.98.6, I find
> JmxCacheBuster::clearJmxCache() does no take effect actually. The related
> code are listed below:
> {code:title=org.apache.hadoop.metrics2.impl.JmxCacheBuster.java|borderStyle=solid}
> // fut is initialized to null
> private static AtomicReference fut = new
> AtomicReference<>(null);
> public static void clearJmxCache() {
> // clearJmxCache return directly when fut is null, which is always true.
> // the actual intent is 'if (future != null && !future.isDone ...)' ?
> ScheduledFuture future = fut.get();
> if ((future == null || (!future.isDone() &&
> future.getDelay(TimeUnit.MILLISECONDS) > 100))) {
> return;
> }
> ..
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15321) Ability to open a HRegion from hdfs snapshot.
[ https://issues.apache.org/jira/browse/HBASE-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-15321: --- Attachment: HBASE-15321.patch Here is the patch for master. > Ability to open a HRegion from hdfs snapshot. > - > > Key: HBASE-15321 > URL: https://issues.apache.org/jira/browse/HBASE-15321 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: churro morales > Fix For: 2.0.0 > > Attachments: HBASE-15321.patch > > > Now that hdfs snapshots are here, we started to run our mapreduce jobs over > hdfs snapshots. The thing is, hdfs snapshots are read-only point-in-time > copies of the file system. Thus we had to modify the section of code that > initialized the region internals in HRegion. We have to skip cleanup of > certain directories if the HRegion is backed by a hdfs snapshot. I have a > patch for trunk with some basic tests if folks are interested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15222) Use less contended classes for metrics
[ https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166693#comment-15166693 ] Hudson commented on HBASE-15222: FAILURE: Integrated in HBase-Trunk_matrix #736 (See [https://builds.apache.org/job/HBase-Trunk_matrix/736/]) HBASE-15222 Use less contended classes for metrics (eclark: rev 630a65825ed9a9c00f72bbfcac0588e1ab0cdd72) * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsSnapshotSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterFilesystemSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/thrift/MetricsThriftServerSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/AgeSnapshot.java * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/metrics/BaseSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableTimeHistogram.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsReplicationSourceSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableHistogram.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/metrics2/MetricHistogram.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsAssignmentManagerSourceImpl.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/FastLongHistogram.java * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/BlockCacheViewTmpl.jamon * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/DynamicMetricsRegistry.java * hbase-hadoop2-compat/pom.xml * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableSizeHistogram.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCacheStats.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsReplicationGlobalSourceSource.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCacheUtil.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal/MetricsWALSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricMutableQuantiles.java * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/BlockCacheTmpl.jamon * hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestFastLongHistogram.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricsExecutorImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/balancer/MetricsBalancerSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/rest/MetricsRESTSourceImpl.java * hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/metrics/TestBaseSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableFastCounter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsReplicationSinkSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/metrics/BaseSource.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableRangeHistogram.java > Use less contended classes for metrics > -- > > Key: HBASE-15222 > URL: https://issues.apache.org/jira/browse/HBASE-15222 > Project: HBase > Issue Type: Bug > Components: metrics >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15222-v1.patch, HBASE-15222-v10.patch, > HBASE-15222-v11.patch, HBASE-15222-v12.patch, HBASE-15222-v13.patch, > HBASE-15222-v2.patch, HBASE-15222-v3.patch, HBASE-15222-v5.patch, > HBASE-15222-v6.patch, HBASE-15222-v8.patch, HBASE-15222-v9.patch, > HBASE-15222.p
[jira] [Commented] (HBASE-15319) clearJmxCache does not take effect actually
[
https://issues.apache.org/jira/browse/HBASE-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166692#comment-15166692
]
Hudson commented on HBASE-15319:
FAILURE: Integrated in HBase-Trunk_matrix #736 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/736/])
HBASE-15319 clearJmxCache does not take effect actually (eclark: rev
a3b4575f700c20167a6ab0b774d8a2c9cd3916af)
*
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/impl/JmxCacheBuster.java
> clearJmxCache does not take effect actually
> ---
>
> Key: HBASE-15319
> URL: https://issues.apache.org/jira/browse/HBASE-15319
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.0.0, 1.2.0, 1.1.0.1, 1.3.0
>Reporter: Hao Lin
>Assignee: Elliott Clark
> Labels: easyfix
> Fix For: 2.0.0, 1.3.0, 1.2.1
>
> Attachments: HBASE-15319.patch
>
>
> When trying to backport HBASE-14166 to 0.98.6, I find
> JmxCacheBuster::clearJmxCache() does no take effect actually. The related
> code are listed below:
> {code:title=org.apache.hadoop.metrics2.impl.JmxCacheBuster.java|borderStyle=solid}
> // fut is initialized to null
> private static AtomicReference fut = new
> AtomicReference<>(null);
> public static void clearJmxCache() {
> // clearJmxCache return directly when fut is null, which is always true.
> // the actual intent is 'if (future != null && !future.isDone ...)' ?
> ScheduledFuture future = fut.get();
> if ((future == null || (!future.isDone() &&
> future.getDelay(TimeUnit.MILLISECONDS) > 100))) {
> return;
> }
> ..
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12133) Add FastLongHistogram for metric computation
[ https://issues.apache.org/jira/browse/HBASE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166673#comment-15166673 ] Hudson commented on HBASE-12133: SUCCESS: Integrated in HBase-1.3-IT #515 (See [https://builds.apache.org/job/HBase-1.3-IT/515/]) HBASE-12133 Add FastLongHistogram for metric computation (Yi Deng) (eclark: rev b4f747f52e24afcf539a313a3bd2243ccf89b6b1) * hbase-common/src/main/java/org/apache/hadoop/hbase/util/AtomicUtils.java * hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestFastLongHistogram.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/FastLongHistogram.java > Add FastLongHistogram for metric computation > > > Key: HBASE-12133 > URL: https://issues.apache.org/jira/browse/HBASE-12133 > Project: HBase > Issue Type: New Feature > Components: metrics >Affects Versions: 0.98.8 >Reporter: Yi Deng >Assignee: Yi Deng >Priority: Minor > Labels: histogram, metrics > Fix For: 2.0.0, 0.99.1 > > Attachments: > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 12133.addendum.txt > > > FastLongHistogram is a thread-safe class that estimate distribution of data > and computes the quantiles. It's useful for computing aggregated metrics like > P99/P95. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15222) Use less contended classes for metrics
[ https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166675#comment-15166675 ] Hudson commented on HBASE-15222: SUCCESS: Integrated in HBase-1.3-IT #515 (See [https://builds.apache.org/job/HBase-1.3-IT/515/]) HBASE-15222 Use less contended classes for metrics (eclark: rev 3352173ec87debdfe115755768c3f9c0ca82bb0e) * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableHistogram.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/BlockCacheViewTmpl.jamon * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsReplicationSinkSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/DynamicMetricsRegistry.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/AgeSnapshot.java * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/BlockCacheTmpl.jamon * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCacheStats.java * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon * hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/metrics/TestBaseSourceImpl.java * hbase-hadoop2-compat/pom.xml * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/metrics/BaseSource.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/metrics2/MetricHistogram.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableFastCounter.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal/MetricsWALSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCacheUtil.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricMutableQuantiles.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableSizeHistogram.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/FastLongHistogram.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsReplicationGlobalSourceSource.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsAssignmentManagerSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableTimeHistogram.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsSnapshotSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/metrics/BaseSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/balancer/MetricsBalancerSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricsExecutorImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/rest/MetricsRESTSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/thrift/MetricsThriftServerSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterFilesystemSourceImpl.java * hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestFastLongHistogram.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableRangeHistogram.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsReplicationSourceSourceImpl.java > Use less contended classes for metrics > -- > > Key: HBASE-15222 > URL: https://issues.apache.org/jira/browse/HBASE-15222 > Project: HBase > Issue Type: Bug > Components: metrics >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15222-v1.patch, HBASE-15222-v10.patch, > HBASE-15222-v11.patch, HBASE-15222-v12.patch, HBASE-15222-v13.patch, > HBASE-15222-v2.patch, HBASE-15222-v3.patch, HBASE-15222-v5.patch, > HBASE-15222-v6.patch, HBASE-15222-v8.patch, HBASE-15222-v9.patch, > HBASE-15222.patch > > > Running the benchmarks now, but it looks like the results are pretty extreme. > The locking
[jira] [Commented] (HBASE-15319) clearJmxCache does not take effect actually
[
https://issues.apache.org/jira/browse/HBASE-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166674#comment-15166674
]
Hudson commented on HBASE-15319:
SUCCESS: Integrated in HBase-1.3-IT #515 (See
[https://builds.apache.org/job/HBase-1.3-IT/515/])
HBASE-15319 clearJmxCache does not take effect actually (eclark: rev
13773254c87994c6cffdb3fbc48521d51c86d8c5)
*
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/impl/JmxCacheBuster.java
> clearJmxCache does not take effect actually
> ---
>
> Key: HBASE-15319
> URL: https://issues.apache.org/jira/browse/HBASE-15319
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.0.0, 1.2.0, 1.1.0.1, 1.3.0
>Reporter: Hao Lin
>Assignee: Elliott Clark
> Labels: easyfix
> Fix For: 2.0.0, 1.3.0, 1.2.1
>
> Attachments: HBASE-15319.patch
>
>
> When trying to backport HBASE-14166 to 0.98.6, I find
> JmxCacheBuster::clearJmxCache() does no take effect actually. The related
> code are listed below:
> {code:title=org.apache.hadoop.metrics2.impl.JmxCacheBuster.java|borderStyle=solid}
> // fut is initialized to null
> private static AtomicReference fut = new
> AtomicReference<>(null);
> public static void clearJmxCache() {
> // clearJmxCache return directly when fut is null, which is always true.
> // the actual intent is 'if (future != null && !future.isDone ...)' ?
> ScheduledFuture future = fut.get();
> if ((future == null || (!future.isDone() &&
> future.getDelay(TimeUnit.MILLISECONDS) > 100))) {
> return;
> }
> ..
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15290) Hbase Rest CheckAndAPI should save other cells along with compared cell
[ https://issues.apache.org/jira/browse/HBASE-15290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166658#comment-15166658 ] Ajith commented on HBASE-15290: --- Linked and provided the patch for the CheckAndDelete also as another patch. Please get both of them in. > Hbase Rest CheckAndAPI should save other cells along with compared cell > --- > > Key: HBASE-15290 > URL: https://issues.apache.org/jira/browse/HBASE-15290 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.1.1 > Environment: Linux and windows >Reporter: Ajith > Labels: easyfix > Attachments: HBASE-15290-checkAndPut.patch, > HBASE-15290-checkAndPut_Comments.patch, checkputfix2.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > Java CheckAndPut API allows users to save Cells (C1..C5) while comparing a > Cell C1. > But in Rest API, even though caller sent multiple cells, hbase rest code is > ignoring all the cells except for compare cell. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15323) Hbase Rest CheckAndDeleteAPi should be able to delete more cells
[ https://issues.apache.org/jira/browse/HBASE-15323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith updated HBASE-15323: -- Attachment: HBASE-15323-Check-And-Delete-Fix.patch Patch tested on Linux one box. > Hbase Rest CheckAndDeleteAPi should be able to delete more cells > > > Key: HBASE-15323 > URL: https://issues.apache.org/jira/browse/HBASE-15323 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.1.1 > Environment: Linux And Windows >Reporter: Ajith > Attachments: HBASE-15323-Check-And-Delete-Fix.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > Java CheckAndDelete API accepts Delete object which can be used to delete (a > cell / cell version / multiple cells / column family or a row), but the rest > api only allows to delete the cell (without any version) > Need to add this capability to rest api. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15290) Hbase Rest CheckAndAPI should save other cells along with compared cell
[ https://issues.apache.org/jira/browse/HBASE-15290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166656#comment-15166656 ] Ajith commented on HBASE-15290: --- CheckAndDelete & CheckAndPut are similar issues. > Hbase Rest CheckAndAPI should save other cells along with compared cell > --- > > Key: HBASE-15290 > URL: https://issues.apache.org/jira/browse/HBASE-15290 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.1.1 > Environment: Linux and windows >Reporter: Ajith > Labels: easyfix > Attachments: HBASE-15290-checkAndPut.patch, > HBASE-15290-checkAndPut_Comments.patch, checkputfix2.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > Java CheckAndPut API allows users to save Cells (C1..C5) while comparing a > Cell C1. > But in Rest API, even though caller sent multiple cells, hbase rest code is > ignoring all the cells except for compare cell. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15323) Hbase Rest CheckAndDeleteAPi should be able to delete more cells
Ajith created HBASE-15323: - Summary: Hbase Rest CheckAndDeleteAPi should be able to delete more cells Key: HBASE-15323 URL: https://issues.apache.org/jira/browse/HBASE-15323 Project: HBase Issue Type: Bug Components: hbase Affects Versions: 1.1.1 Environment: Linux And Windows Reporter: Ajith Java CheckAndDelete API accepts Delete object which can be used to delete (a cell / cell version / multiple cells / column family or a row), but the rest api only allows to delete the cell (without any version) Need to add this capability to rest api. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15322) HBase 1.1.3 crashing
[ https://issues.apache.org/jira/browse/HBASE-15322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166654#comment-15166654 ] Anant Sharma commented on HBASE-15322: -- Dont have any other exception before that in the HMaster log but have exception in HRegionServer log: __ 2016-02-25 03:17:07,281 ERROR [main] regionserver.HRegionServerCommandLine: Region server exiting java.lang.RuntimeException: Failed construction of Regionserver: class org.apache.hadoop.hbase.regionserver.HRegionServer at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2635) at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:64) at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2650) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2633) ... 5 more Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.util.Bytes$LexicographicalComparerHolder$UnsafeComparer at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:899) at org.apache.hadoop.hbase.KeyValue.createByteArray(KeyValue.java:1082) at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:652) at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:580) at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:483) at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:370) at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:267) at org.apache.hadoop.hbase.HConstants.(HConstants.java:978) at org.apache.hadoop.hbase.HTableDescriptor.(HTableDescriptor.java:1488) at org.apache.hadoop.hbase.util.FSTableDescriptors.(FSTableDescriptors.java:124) at org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:570) ... 10 more __ This is also caused by the same NoClassDefFoundError as that of HMaster > HBase 1.1.3 crashing > > > Key: HBASE-15322 > URL: https://issues.apache.org/jira/browse/HBASE-15322 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.1.3 > Environment: OS: Ubuntu 14.04/Ubuntu 15.10 > JDK: OpenJDK8/OpenJDK9 >Reporter: Anant Sharma > > HBase crashes in standalone mode with the following log: > __ > 2016-02-24 22:38:37,578 ERROR [main] master.HMasterCommandLine: Master exiting > java.lang.RuntimeException: Failed construction of Master: class > org.apache.hadoop.hbase.master.HMaster > at > org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2341) > at > org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:233) > at > org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) > at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2355) > Caused by: java.lang.NoClassDefFoundError: Could not initialize class > org.apache.hadoop.hbase.util.Bytes$LexicographicalComparerHolder$UnsafeComparer > at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:899) > at > org.apache.hadoop.hbase.KeyValue.createByteArray(KeyValue.java:1082) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:652) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:580) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:483) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:370) > at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:267) > at org.apache.hadoop.hbase.HConstants.(HConstants.java:978) > at > org.apache.hadoop.hbase.HTableDescriptor.
[jira] [Commented] (HBASE-15187) Integrate CSRF prevention filter to REST gateway
[
https://issues.apache.org/jira/browse/HBASE-15187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166647#comment-15166647
]
Hadoop QA commented on HBASE-15187:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 29s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
37s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 42s
{color} | {color:red} hbase-rest in master has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 32s
{color} | {color:red} Patch generated 1 new checkstyle issues in hbase-rest
(total was 30, now 30). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
24m 18s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 48s
{color} | {color:green} hbase-common in the patch passed with JDK v1.8.0_72.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 7s
{color} | {color:green} hbase-rest in the patch passed with JDK v1.8.0_72.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 41s
{color} | {color:green} hbase-common in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 2s
{color} | {color:green} hbase-rest in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {
[jira] [Commented] (HBASE-15312) Update the dependences of pom for mini cluster in HBase Book
[ https://issues.apache.org/jira/browse/HBASE-15312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166642#comment-15166642 ] stack commented on HBASE-15312: --- nvm [~liushaohui] Pilot error on my end. > Update the dependences of pom for mini cluster in HBase Book > > > Key: HBASE-15312 > URL: https://issues.apache.org/jira/browse/HBASE-15312 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15312-trunk-v1.diff, HBASE-15312-trunk-v2.diff > > > In HBase book, the dependences of pom for mini cluster is outdated after > version 0.96. > See: > http://hbase.apache.org/book.html#_integration_testing_with_an_hbase_mini_cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15322) HBase 1.1.3 crashing
[
https://issues.apache.org/jira/browse/HBASE-15322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166641#comment-15166641
]
Duo Zhang commented on HBASE-15322:
---
{{NoClassDefFoundError}} could happen when you have that class on your
classpath if there are exceptions while loading the class. Is there any other
exception before the {{NoClassDefFoundError}} in your log?
> HBase 1.1.3 crashing
>
>
> Key: HBASE-15322
> URL: https://issues.apache.org/jira/browse/HBASE-15322
> Project: HBase
> Issue Type: Bug
> Components: hbase
>Affects Versions: 1.1.3
> Environment: OS: Ubuntu 14.04/Ubuntu 15.10
> JDK: OpenJDK8/OpenJDK9
>Reporter: Anant Sharma
>
> HBase crashes in standalone mode with the following log:
> __
> 2016-02-24 22:38:37,578 ERROR [main] master.HMasterCommandLine: Master exiting
> java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMaster
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2341)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:233)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2355)
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.hbase.util.Bytes$LexicographicalComparerHolder$UnsafeComparer
> at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:899)
> at
> org.apache.hadoop.hbase.KeyValue.createByteArray(KeyValue.java:1082)
> at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:652)
> at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:580)
> at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:483)
> at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:370)
> at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:267)
> at org.apache.hadoop.hbase.HConstants.(HConstants.java:978)
> at
> org.apache.hadoop.hbase.HTableDescriptor.(HTableDescriptor.java:1488)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.(FSTableDescriptors.java:124)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:570)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:365)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2336)
> __
> The class is in the hbase-common.jar and its there in the classpath as can be
> seen from the log:
> _
> 2016-02-24 22:38:32,538 INFO [main] util.ServerCommandLine:
> env:CLASSPATH=/home/hduser/hbase/hbase-1.1.3:/home/hduser/hbase/hbase-1.1.3/lib/activation-1.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/aopalliance-1.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/apacheds-i18n-2.0.0-M15.jar:/home/hduser/hbase/hbase-1.1.3/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/hduser/hbase/hbase-1.1.3/lib/api-asn1-api-1.0.0-M20.jar:/home/hduser/hbase/hbase-1.1.3/lib/api-util-1.0.0-M20.jar:/home/hduser/hbase/hbase-1.1.3/lib/asm-3.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/avro-1.7.4.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-beanutils-1.7.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-beanutils-core-1.8.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-cli-1.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-codec-1.9.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-collections-3.2.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-compress-1.4.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-configuration-1.6.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-daemon-1.0.13.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-digester-1.8.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-el-1.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-httpclient-3.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-io-2.4.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-lang-2.6.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-logging-1.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-math-2.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-math3-3.1.1.jar:/home/hduser/
[jira] [Commented] (HBASE-15312) Update the dependences of pom for mini cluster in HBase Book
[ https://issues.apache.org/jira/browse/HBASE-15312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166639#comment-15166639 ] stack commented on HBASE-15312: --- Addendum doesn't apply for me [~liushaohui]. For you? Thanks. > Update the dependences of pom for mini cluster in HBase Book > > > Key: HBASE-15312 > URL: https://issues.apache.org/jira/browse/HBASE-15312 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15312-trunk-v1.diff, HBASE-15312-trunk-v2.diff > > > In HBase book, the dependences of pom for mini cluster is outdated after > version 0.96. > See: > http://hbase.apache.org/book.html#_integration_testing_with_an_hbase_mini_cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15265) Implement an asynchronous FSHLog
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15265: -- Status: Patch Available (was: Open) > Implement an asynchronous FSHLog > > > Key: HBASE-15265 > URL: https://issues.apache.org/jira/browse/HBASE-15265 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-15265.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15265) Implement an asynchronous FSHLog
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15265: -- Fix Version/s: 2.0.0 Component/s: wal > Implement an asynchronous FSHLog > > > Key: HBASE-15265 > URL: https://issues.apache.org/jira/browse/HBASE-15265 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-15265.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15265) Implement an asynchronous FSHLog
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15265: -- Attachment: HBASE-15265.patch The patch is big so put it here for reviewing first. Will work on the create and shutdown issues. > Implement an asynchronous FSHLog > > > Key: HBASE-15265 > URL: https://issues.apache.org/jira/browse/HBASE-15265 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-15265.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15302) Reenable the other tests disabled by HBASE-14678
[ https://issues.apache.org/jira/browse/HBASE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-15302: -- Attachment: HBASE-15302-branch-1-v1.txt Remove TestAssignmentManager temporarily to see if others can pass tests. If so we can push them first > Reenable the other tests disabled by HBASE-14678 > > > Key: HBASE-15302 > URL: https://issues.apache.org/jira/browse/HBASE-15302 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.3.0, 1.2.1 > > Attachments: HBASE-15302-branch-1-v1.txt, HBASE-15302-v1.txt, > HBASE-15302-v1.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15312) Update the dependences of pom for mini cluster in HBase Book
[ https://issues.apache.org/jira/browse/HBASE-15312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Shaohui updated HBASE-15312: Attachment: HBASE-15312-trunk-v2.diff Update the patch according to [~eclark]'s suggestion~ > Update the dependences of pom for mini cluster in HBase Book > > > Key: HBASE-15312 > URL: https://issues.apache.org/jira/browse/HBASE-15312 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15312-trunk-v1.diff, HBASE-15312-trunk-v2.diff > > > In HBase book, the dependences of pom for mini cluster is outdated after > version 0.96. > See: > http://hbase.apache.org/book.html#_integration_testing_with_an_hbase_mini_cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15312) Update the dependences of pom for mini cluster in HBase Book
[
https://issues.apache.org/jira/browse/HBASE-15312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166587#comment-15166587
]
Liu Shaohui commented on HBASE-15312:
-
[~eclark]
{quote}
hbase-testing-util is supposed to be the answer to all of this ? Does that not
work ?
{quote}
Yes. After test, hbase-testing-util works for the mini cluster tests.
Thanks for your reminding.
[~appy] [~saint@gmail.com]
I will update the patch according to [~eclark]'s suggestion.
> Update the dependences of pom for mini cluster in HBase Book
>
>
> Key: HBASE-15312
> URL: https://issues.apache.org/jira/browse/HBASE-15312
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15312-trunk-v1.diff
>
>
> In HBase book, the dependences of pom for mini cluster is outdated after
> version 0.96.
> See:
> http://hbase.apache.org/book.html#_integration_testing_with_an_hbase_mini_cluster
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Resolution: Fixed Hadoop Flags: Reviewed Release Note: Implement a fan-out asynchronous DFSOutputStream for implementing new WAL writer. Status: Resolved (was: Patch Available) Pushed to master. Thanks [~stack] for the long reviewing. > Implement a fan out HDFS OutputStream > - > > Key: HBASE-15264 > URL: https://issues.apache.org/jira/browse/HBASE-15264 > Project: HBase > Issue Type: Sub-task > Components: util, wal >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-15264-v1.patch, HBASE-15264-v2.patch, > HBASE-15264-v3.patch, HBASE-15264-v4.patch, HBASE-15264-v5.patch, > HBASE-15264-v6.patch, HBASE-15264-v7.patch, HBASE-15264-v8.patch, > HBASE-15264.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14801) Enhance the Spark-HBase connector catalog with json format
[
https://issues.apache.org/jira/browse/HBASE-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166579#comment-15166579
]
Hadoop QA commented on HBASE-14801:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 42s
{color} | {color:red} hbase-spark in master has 66 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
29s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
29s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
3s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 34s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 53s
{color} | {color:red} hbase-spark introduced 7 new FindBugs issues. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 5s
{color} | {color:red} hbase-spark-jdk1.8.0_72 with JDK v1.8.0_72 generated 1
new issues (was 15, now 16). {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 20s
{color} | {color:red} hbase-spark-jdk1.7.0_95 with JDK v1.7.0_95 generated 1
new issues (was 15, now 16). {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} |
[jira] [Created] (HBASE-15322) HBase 1.1.3 crashing
Anant Sharma created HBASE-15322: Summary: HBase 1.1.3 crashing Key: HBASE-15322 URL: https://issues.apache.org/jira/browse/HBASE-15322 Project: HBase Issue Type: Bug Components: hbase Affects Versions: 1.1.3 Environment: OS: Ubuntu 14.04/Ubuntu 15.10 JDK: OpenJDK8/OpenJDK9 Reporter: Anant Sharma HBase crashes in standalone mode with the following log: __ 2016-02-24 22:38:37,578 ERROR [main] master.HMasterCommandLine: Master exiting java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2341) at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:233) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2355) Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.util.Bytes$LexicographicalComparerHolder$UnsafeComparer at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:899) at org.apache.hadoop.hbase.KeyValue.createByteArray(KeyValue.java:1082) at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:652) at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:580) at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:483) at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:370) at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:267) at org.apache.hadoop.hbase.HConstants.(HConstants.java:978) at org.apache.hadoop.hbase.HTableDescriptor.(HTableDescriptor.java:1488) at org.apache.hadoop.hbase.util.FSTableDescriptors.(FSTableDescriptors.java:124) at org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:570) at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:365) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2336) __ The class is in the hbase-common.jar and its there in the classpath as can be seen from the log: _ 2016-02-24 22:38:32,538 INFO [main] util.ServerCommandLine: env:CLASSPATH=/home/hduser/hbase/hbase-1.1.3:/home/hduser/hbase/hbase-1.1.3/lib/activation-1.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/aopalliance-1.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/apacheds-i18n-2.0.0-M15.jar:/home/hduser/hbase/hbase-1.1.3/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/hduser/hbase/hbase-1.1.3/lib/api-asn1-api-1.0.0-M20.jar:/home/hduser/hbase/hbase-1.1.3/lib/api-util-1.0.0-M20.jar:/home/hduser/hbase/hbase-1.1.3/lib/asm-3.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/avro-1.7.4.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-beanutils-1.7.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-beanutils-core-1.8.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-cli-1.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-codec-1.9.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-collections-3.2.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-compress-1.4.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-configuration-1.6.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-daemon-1.0.13.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-digester-1.8.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-el-1.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-httpclient-3.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-io-2.4.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-lang-2.6.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-logging-1.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-math-2.2.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-math3-3.1.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/commons-net-3.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/disruptor-3.3.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/findbugs-annotations-1.3.9-1.jar:/home/hduser/hbase/hbase-1.1.3/lib/guava-12.0.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/guice-3.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/guice-servlet-3.0.jar:/home/hduser/hbase/hbase-1.1.3/lib/hadoop-annotations-2.5.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/hadoop-auth-2.5.1.jar:/home/hduser/hbase/hbase-1.1.3/lib/hadoop-client-2.5.1.ja
[jira] [Commented] (HBASE-15222) Use less contended classes for metrics
[ https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166573#comment-15166573 ] Hudson commented on HBASE-15222: FAILURE: Integrated in HBase-1.3 #569 (See [https://builds.apache.org/job/HBase-1.3/569/]) HBASE-15222 Use less contended classes for metrics (eclark: rev 3352173ec87debdfe115755768c3f9c0ca82bb0e) * hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestFastLongHistogram.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableFastCounter.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableTimeHistogram.java * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/thrift/MetricsThriftServerSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/metrics/BaseSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableHistogram.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsSnapshotSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal/MetricsWALSourceImpl.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/metrics2/MetricHistogram.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsReplicationSinkSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableRangeHistogram.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterFilesystemSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricsExecutorImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java * hbase-hadoop2-compat/pom.xml * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/BlockCacheViewTmpl.jamon * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsAssignmentManagerSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/AgeSnapshot.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsReplicationSourceSourceImpl.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/metrics/BaseSource.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/FastLongHistogram.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCacheUtil.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableSizeHistogram.java * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/BlockCacheTmpl.jamon * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricMutableQuantiles.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java * hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/metrics/TestBaseSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/balancer/MetricsBalancerSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/DynamicMetricsRegistry.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCacheStats.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsReplicationGlobalSourceSource.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/rest/MetricsRESTSourceImpl.java > Use less contended classes for metrics > -- > > Key: HBASE-15222 > URL: https://issues.apache.org/jira/browse/HBASE-15222 > Project: HBase > Issue Type: Bug > Components: metrics >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15222-v1.patch, HBASE-15222-v10.patch, > HBASE-15222-v11.patch, HBASE-15222-v12.patch, HBASE-15222-v13.patch, > HBASE-15222-v2.patch, HBASE-15222-v3.patch, HBASE-15222-v5.patch, > HBASE-15222-v6.patch, HBASE-15222-v8.patch, HBASE-15222-v9.patch, > HBASE-15222.patch > > > Running the benchmarks now, but it looks like the results are pretty extreme. > The locking in our
[jira] [Commented] (HBASE-12133) Add FastLongHistogram for metric computation
[ https://issues.apache.org/jira/browse/HBASE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166572#comment-15166572 ] Hudson commented on HBASE-12133: FAILURE: Integrated in HBase-1.3 #569 (See [https://builds.apache.org/job/HBase-1.3/569/]) HBASE-12133 Add FastLongHistogram for metric computation (Yi Deng) (eclark: rev b4f747f52e24afcf539a313a3bd2243ccf89b6b1) * hbase-common/src/main/java/org/apache/hadoop/hbase/util/FastLongHistogram.java * hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestFastLongHistogram.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/AtomicUtils.java > Add FastLongHistogram for metric computation > > > Key: HBASE-12133 > URL: https://issues.apache.org/jira/browse/HBASE-12133 > Project: HBase > Issue Type: New Feature > Components: metrics >Affects Versions: 0.98.8 >Reporter: Yi Deng >Assignee: Yi Deng >Priority: Minor > Labels: histogram, metrics > Fix For: 2.0.0, 0.99.1 > > Attachments: > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 12133.addendum.txt > > > FastLongHistogram is a thread-safe class that estimate distribution of data > and computes the quantiles. It's useful for computing aggregated metrics like > P99/P95. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15187) Integrate CSRF prevention filter to REST gateway
[ https://issues.apache.org/jira/browse/HBASE-15187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15187: --- Attachment: HBASE-15187.v10.patch Patch v10 corrects two checkstyle warnings. Method length warning was due to pre-existing long method. > Integrate CSRF prevention filter to REST gateway > > > Key: HBASE-15187 > URL: https://issues.apache.org/jira/browse/HBASE-15187 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: HBASE-15187.v1.patch, HBASE-15187.v10.patch, > HBASE-15187.v2.patch, HBASE-15187.v3.patch, HBASE-15187.v4.patch, > HBASE-15187.v5.patch, HBASE-15187.v6.patch, HBASE-15187.v7.patch, > HBASE-15187.v8.patch, HBASE-15187.v9.patch > > > HADOOP-12691 introduced a filter in Hadoop Common to help REST APIs guard > against cross-site request forgery attacks. > This issue tracks the integration of that filter into HBase REST gateway. > From REST section of refguide: > To delete a table, use a DELETE request with the /schema endpoint: > http://example.com:8000/schema > Suppose an attacker hosts a malicious web form on a domain under his control. > The form uses the DELETE action targeting a REST URL. Through social > engineering, the attacker tricks an authenticated user into accessing the > form and submitting it. > The browser sends the HTTP DELETE request to the REST gateway. > At REST gateway, the call is executed and user table is dropped -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[
https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166529#comment-15166529
]
Francis Liu commented on HBASE-6721:
Attached WIP rebased patch which addresses the following:
{quote}
Yeah, address the groups naming thing.
{quote}
Partially done. I renamed the main proto classes and package name to Group ->
RSGroup. Should I just blindly prepend all entities with Group in it to RSGroup?
{quote}
Address the ServerName questions, breaking compat is not an option.
{quote}
Waiting on response from [~eclark]
{quote}
Don't go around the Metrics to get to jmx
{quote}
Waiting on response from [~eclark]
{quote}
Don't create a new protobuf class when not needed
{quote}
Done
{quote}
Move it into a different module with a profile that doesn't include it.
{quote}
Partially done. Most of the classes are in a different module. Need to address
circular dependecy between hbase-rsgroup and hbase-server. Since the
MasterObserver CP has hooks into the GroupAdminEndpoint. Pom file needs a bit
of streamlining as well.
> RegionServer Group based Assignment
> ---
>
> Key: HBASE-6721
> URL: https://issues.apache.org/jira/browse/HBASE-6721
> Project: HBase
> Issue Type: New Feature
>Reporter: Francis Liu
>Assignee: Francis Liu
> Labels: hbase-6721
> Attachments: 6721-master-webUI.patch, HBASE-6721
> GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf,
> HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf,
> HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch,
> HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch,
> HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch,
> HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch,
> HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch,
> HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch,
> HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch,
> HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch,
> HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch,
> HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg,
> hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch,
> hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch,
> hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch,
> hbase-6721-v25.patch, hbase-6721-v26_draft1.patch, immediateAssignments
> Sequence Diagram.svg, randomAssignment Sequence Diagram.svg, retainAssignment
> Sequence Diagram.svg, roundRobinAssignment Sequence Diagram.svg
>
>
> In multi-tenant deployments of HBase, it is likely that a RegionServer will
> be serving out regions from a number of different tables owned by various
> client applications. Being able to group a subset of running RegionServers
> and assign specific tables to it, provides a client application a level of
> isolation and resource allocation.
> The proposal essentially is to have an AssignmentManager which is aware of
> RegionServer groups and assigns tables to region servers based on groupings.
> Load balancing will occur on a per group basis as well.
> This is essentially a simplification of the approach taken in HBASE-4120. See
> attached document.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15290) Hbase Rest CheckAndAPI should save other cells along with compared cell
[
https://issues.apache.org/jira/browse/HBASE-15290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166521#comment-15166521
]
Enis Soztutar commented on HBASE-15290:
---
bq. Enis, I would enhance CheckAndDelete as separate Jira as its is different
api enhancement. Can we get this come completed?
Fair enough. Please open a jira and link it here.
We should not add this header since it already exists. I can address it on
commit.
{code}
+ * Licensed to the Apache Software Foundation (ASF) under one
{code}
Let me kick a hadoopqa run.
> Hbase Rest CheckAndAPI should save other cells along with compared cell
> ---
>
> Key: HBASE-15290
> URL: https://issues.apache.org/jira/browse/HBASE-15290
> Project: HBase
> Issue Type: Bug
> Components: hbase
>Affects Versions: 1.1.1
> Environment: Linux and windows
>Reporter: Ajith
> Labels: easyfix
> Attachments: HBASE-15290-checkAndPut.patch,
> HBASE-15290-checkAndPut_Comments.patch, checkputfix2.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> Java CheckAndPut API allows users to save Cells (C1..C5) while comparing a
> Cell C1.
> But in Rest API, even though caller sent multiple cells, hbase rest code is
> ignoring all the cells except for compare cell.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15187) Integrate CSRF prevention filter to REST gateway
[
https://issues.apache.org/jira/browse/HBASE-15187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166512#comment-15166512
]
Hadoop QA commented on HBASE-15187:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
3s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 47s
{color} | {color:red} hbase-rest in master has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 33s
{color} | {color:red} Patch generated 3 new checkstyle issues in hbase-rest
(total was 30, now 32). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 0s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 33s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 43s
{color} | {color:green} hbase-common in the patch passed with JDK v1.8.0_72.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 40s
{color} | {color:green} hbase-rest in the patch passed with JDK v1.8.0_72.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 46s
{color} | {color:green} hbase-common in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 40s
{color} | {color:green} hbase-rest in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} |
[jira] [Updated] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-6721: --- Attachment: hbase-6721-v26_draft1.patch > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch, > hbase-6721-v25.patch, hbase-6721-v26_draft1.patch, immediateAssignments > Sequence Diagram.svg, randomAssignment Sequence Diagram.svg, retainAssignment > Sequence Diagram.svg, roundRobinAssignment Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15295) MutateTableAccess.multiMutate() does not get high priority causing a deadlock
[
https://issues.apache.org/jira/browse/HBASE-15295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-15295:
--
Attachment: hbase-15295_v2.patch
bq. why not add controller for access control method in ProtobufUtil?
I think I've missed it when going over methods in Admin. Since they are called
from AccessControlClient. Anyway, patch v2 addresses that.
Attaching v2 for hadoopqa.
> MutateTableAccess.multiMutate() does not get high priority causing a deadlock
> -
>
> Key: HBASE-15295
> URL: https://issues.apache.org/jira/browse/HBASE-15295
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.4
>
> Attachments: hbase-15295_v1.patch, hbase-15295_v1.patch,
> hbase-15295_v2.patch
>
>
> We have seen this in a cluster with Phoenix secondary indexes leading to a
> deadlock. All handlers are busy waiting on the index updates to finish:
> {code}
> "B.defaultRpcServer.handler=50,queue=0,port=16020" #91 daemon prio=5
> os_prio=0 tid=0x7f29f64ba000 nid=0xab51 waiting on condition
> [0x7f29a8762000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x000124f1d5c8> (a
> com.google.common.util.concurrent.AbstractFuture$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> at
> com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:275)
> at
> com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:111)
> at
> org.apache.phoenix.hbase.index.parallel.BaseTaskRunner.submit(BaseTaskRunner.java:66)
> at
> org.apache.phoenix.hbase.index.parallel.BaseTaskRunner.submitUninterruptible(BaseTaskRunner.java:99)
> at
> org.apache.phoenix.hbase.index.write.ParallelWriterIndexCommitter.write(ParallelWriterIndexCommitter.java:194)
> at
> org.apache.phoenix.hbase.index.write.IndexWriter.write(IndexWriter.java:179)
> at
> org.apache.phoenix.hbase.index.write.IndexWriter.writeAndKillYourselfOnFailure(IndexWriter.java:144)
> at
> org.apache.phoenix.hbase.index.write.IndexWriter.writeAndKillYourselfOnFailure(IndexWriter.java:134)
> at
> org.apache.phoenix.hbase.index.Indexer.doPostWithExceptions(Indexer.java:457)
> at org.apache.phoenix.hbase.index.Indexer.doPost(Indexer.java:406)
> at
> org.apache.phoenix.hbase.index.Indexer.postBatchMutate(Indexer.java:401)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$36.call(RegionCoprocessorHost.java:1006)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1673)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1748)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1705)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postBatchMutate(RegionCoprocessorHost.java:1002)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3162)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2801)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2743)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:692)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:654)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2031)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> And the index region is trying to split, and is trying to do a meta update:
> {code}
> "regionserver//10.132.70.191:16020-splits-1454693389669" #1779
> prio=5 os_prio=
[jira] [Commented] (HBASE-14983) Create metrics for per block type hit/miss ratios
[
https://issues.apache.org/jira/browse/HBASE-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166497#comment-15166497
]
Hadoop QA commented on HBASE-14983:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 23s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 12s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m
6s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 12s
{color} | {color:red} hbase-external-blockcache in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 11s
{color} | {color:red} hbase-hadoop2-compat in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 17s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 11s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 11s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 15s
{color} | {color:red} Patch generated 1 new checkstyle issues in
hbase-hadoop2-compat (total was 3, now 3). {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m 1s
{color} | {color:red} Patch generated 3 new checkstyle issues in hbase-server
(total was 171, now 173). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 40s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 10s
{color} | {color:red} hbase-server introduced 2 new FindBugs issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 14s
{color} | {color:green} hbase-external-blockcache in the patch passed with JDK
v1.8.0_72. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s
{color} | {color:green} hbase-hadoop2-compat in the patch passed with JDK
v1.8.0_72. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 19s
{color} | {color:green} hbase-hadoop-compat in the patch passed with JDK
v1.8.0_72. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 17m 14s {color}
| {color:red} hbase-server in the patch failed with J
[jira] [Commented] (HBASE-14801) Enhance the Spark-HBase connector catalog with json format
[ https://issues.apache.org/jira/browse/HBASE-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166481#comment-15166481 ] Ted Malaska commented on HBASE-14801: - I'm having a full day I will be able to look tomorrow. But before we more forward with this. I think I would like to see at least two more committers review this patch. Because this patch includes to components. 1. Is if the code is correct and tested, 2. Is the new style of defining tables what we want to commit to long term. Code review I can do. But the style review I would like to see a couple more people give their say. Because if we make this change I would like to not change it again in the future. > Enhance the Spark-HBase connector catalog with json format > -- > > Key: HBASE-14801 > URL: https://issues.apache.org/jira/browse/HBASE-14801 > Project: HBase > Issue Type: Improvement >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Attachments: HBASE-14801-1.patch, HBASE-14801-2.patch, > HBASE-14801-3.patch, HBASE-14801-4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166467#comment-15166467 ] Elliott Clark commented on HBASE-15136: --- Don't implement equals if you're just going to throw. The default equals should be just fine in this case. Other than that +1 > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15312) Update the dependences of pom for mini cluster in HBase Book
[ https://issues.apache.org/jira/browse/HBASE-15312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166445#comment-15166445 ] Hudson commented on HBASE-15312: FAILURE: Integrated in HBase-Trunk_matrix #735 (See [https://builds.apache.org/job/HBase-Trunk_matrix/735/]) HBASE-15312 Update the dependences of pom for mini cluster in HBase Book (stack: rev 2a306437aaccb99ff333ab41c7165333994eba48) * src/main/asciidoc/_chapters/unit_testing.adoc > Update the dependences of pom for mini cluster in HBase Book > > > Key: HBASE-15312 > URL: https://issues.apache.org/jira/browse/HBASE-15312 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15312-trunk-v1.diff > > > In HBase book, the dependences of pom for mini cluster is outdated after > version 0.96. > See: > http://hbase.apache.org/book.html#_integration_testing_with_an_hbase_mini_cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15310) hbase-spark module has compilation failures with clover profile
[
https://issues.apache.org/jira/browse/HBASE-15310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166444#comment-15166444
]
Hudson commented on HBASE-15310:
FAILURE: Integrated in HBase-Trunk_matrix #735 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/735/])
HBASE-15310 hbase-spark module has compilation failures with clover (jmhsieh:
rev 20e14f449a9d5ba052ef6250c08ee1e4c558ccf2)
* hbase-spark/pom.xml
> hbase-spark module has compilation failures with clover profile
> ---
>
> Key: HBASE-15310
> URL: https://issues.apache.org/jira/browse/HBASE-15310
> Project: HBase
> Issue Type: Bug
> Components: spark, test
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: hbase-15310.patch, hbase-15310.v2.patch
>
>
> running with a clover profile enabled will fail due to cross compilation
> ordering issues with the hbase-spark module.
> {code}
> 21:07:47 [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile
> (default-compile) on project hbase-spark: Compilation failure: Compilation
> failure:
> 21:07:47 [ERROR]
> /data/jenkins/workspace/CDH5.7.0-HBase-1.2.0-Clover/hbase-spark/target/clover/src-instrumented/org/apache/hadoop/hbase/spark/example/hbasecontext/JavaHBaseBulkDeleteExample.java:[23,36]
> error: cannot find symbol
> 21:07:47 [ERROR] symbol: class JavaHBaseContext
> 21:07:47 [ERROR] location: package org.apache.hadoop.hbase.spark
> 21:07:47 [ERROR]
> /data/jenkins/workspace/CDH5.7.0-HBase-1.2.0-Clover/hbase-spark/target/clover/src-instrumented/org/apache/hadoop/hbase/spark/example/hbasecontext/JavaHBaseDistributedScan.java:[27,36]
> error: cannot find symbol
> (many classes)
> {code}
> Apparently this is a known issue and this page shows a remedy.
>
> https://confluence.atlassian.com/display/CLOVERKB/Java-+Scala+cross-compilation+error+-+cannot+find+symbol
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15290) Hbase Rest CheckAndAPI should save other cells along with compared cell
[ https://issues.apache.org/jira/browse/HBASE-15290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166437#comment-15166437 ] Ajith commented on HBASE-15290: --- Enis, I would enhance CheckAndDelete as separate Jira as its is different api enhancement. Can we get this come completed? > Hbase Rest CheckAndAPI should save other cells along with compared cell > --- > > Key: HBASE-15290 > URL: https://issues.apache.org/jira/browse/HBASE-15290 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.1.1 > Environment: Linux and windows >Reporter: Ajith > Labels: easyfix > Attachments: HBASE-15290-checkAndPut.patch, > HBASE-15290-checkAndPut_Comments.patch, checkputfix2.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > Java CheckAndPut API allows users to save Cells (C1..C5) while comparing a > Cell C1. > But in Rest API, even though caller sent multiple cells, hbase rest code is > ignoring all the cells except for compare cell. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15319) clearJmxCache does not take effect actually
[
https://issues.apache.org/jira/browse/HBASE-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Elliott Clark updated HBASE-15319:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks for the review.
> clearJmxCache does not take effect actually
> ---
>
> Key: HBASE-15319
> URL: https://issues.apache.org/jira/browse/HBASE-15319
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.0.0, 1.2.0, 1.1.0.1, 1.3.0
>Reporter: Hao Lin
>Assignee: Elliott Clark
> Labels: easyfix
> Fix For: 2.0.0, 1.3.0, 1.2.1
>
> Attachments: HBASE-15319.patch
>
>
> When trying to backport HBASE-14166 to 0.98.6, I find
> JmxCacheBuster::clearJmxCache() does no take effect actually. The related
> code are listed below:
> {code:title=org.apache.hadoop.metrics2.impl.JmxCacheBuster.java|borderStyle=solid}
> // fut is initialized to null
> private static AtomicReference fut = new
> AtomicReference<>(null);
> public static void clearJmxCache() {
> // clearJmxCache return directly when fut is null, which is always true.
> // the actual intent is 'if (future != null && !future.isDone ...)' ?
> ScheduledFuture future = fut.get();
> if ((future == null || (!future.isDone() &&
> future.getDelay(TimeUnit.MILLISECONDS) > 100))) {
> return;
> }
> ..
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14801) Enhance the Spark-HBase connector catalog with json format
[ https://issues.apache.org/jira/browse/HBASE-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166418#comment-15166418 ] Zhan Zhang commented on HBASE-14801: [~ted.m] Can you help to review the patch as well, and provide your feedback? Thanks for the help. > Enhance the Spark-HBase connector catalog with json format > -- > > Key: HBASE-14801 > URL: https://issues.apache.org/jira/browse/HBASE-14801 > Project: HBase > Issue Type: Improvement >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Attachments: HBASE-14801-1.patch, HBASE-14801-2.patch, > HBASE-14801-3.patch, HBASE-14801-4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14801) Enhance the Spark-HBase connector catalog with json format
[ https://issues.apache.org/jira/browse/HBASE-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166413#comment-15166413 ] Zhan Zhang commented on HBASE-14801: [~ted_yu] The latest patch attached to review board. I quickly go through the findbugs, and they are historical. The findbugs seems to be false positive. I have moved to the right package. Due to spark package access issue, the file has to be located in spark namespace. > Enhance the Spark-HBase connector catalog with json format > -- > > Key: HBASE-14801 > URL: https://issues.apache.org/jira/browse/HBASE-14801 > Project: HBase > Issue Type: Improvement >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Attachments: HBASE-14801-1.patch, HBASE-14801-2.patch, > HBASE-14801-3.patch, HBASE-14801-4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15321) Ability to open a HRegion from hdfs snapshot.
[ https://issues.apache.org/jira/browse/HBASE-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166405#comment-15166405 ] Ted Yu commented on HBASE-15321: Please show your code. > Ability to open a HRegion from hdfs snapshot. > - > > Key: HBASE-15321 > URL: https://issues.apache.org/jira/browse/HBASE-15321 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: churro morales > Fix For: 2.0.0 > > > Now that hdfs snapshots are here, we started to run our mapreduce jobs over > hdfs snapshots. The thing is, hdfs snapshots are read-only point-in-time > copies of the file system. Thus we had to modify the section of code that > initialized the region internals in HRegion. We have to skip cleanup of > certain directories if the HRegion is backed by a hdfs snapshot. I have a > patch for trunk with some basic tests if folks are interested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15319) clearJmxCache does not take effect actually
[
https://issues.apache.org/jira/browse/HBASE-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166402#comment-15166402
]
Ted Yu commented on HBASE-15319:
+1
> clearJmxCache does not take effect actually
> ---
>
> Key: HBASE-15319
> URL: https://issues.apache.org/jira/browse/HBASE-15319
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.0.0, 1.2.0, 1.1.0.1, 1.3.0
>Reporter: Hao Lin
>Assignee: Elliott Clark
> Labels: easyfix
> Fix For: 2.0.0, 1.3.0, 1.2.1
>
> Attachments: HBASE-15319.patch
>
>
> When trying to backport HBASE-14166 to 0.98.6, I find
> JmxCacheBuster::clearJmxCache() does no take effect actually. The related
> code are listed below:
> {code:title=org.apache.hadoop.metrics2.impl.JmxCacheBuster.java|borderStyle=solid}
> // fut is initialized to null
> private static AtomicReference fut = new
> AtomicReference<>(null);
> public static void clearJmxCache() {
> // clearJmxCache return directly when fut is null, which is always true.
> // the actual intent is 'if (future != null && !future.isDone ...)' ?
> ScheduledFuture future = fut.get();
> if ((future == null || (!future.isDone() &&
> future.getDelay(TimeUnit.MILLISECONDS) > 100))) {
> return;
> }
> ..
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15321) Ability to open a HRegion from hdfs snapshot.
churro morales created HBASE-15321: -- Summary: Ability to open a HRegion from hdfs snapshot. Key: HBASE-15321 URL: https://issues.apache.org/jira/browse/HBASE-15321 Project: HBase Issue Type: New Feature Affects Versions: 2.0.0 Reporter: churro morales Fix For: 2.0.0 Now that hdfs snapshots are here, we started to run our mapreduce jobs over hdfs snapshots. The thing is, hdfs snapshots are read-only point-in-time copies of the file system. Thus we had to modify the section of code that initialized the region internals in HRegion. We have to skip cleanup of certain directories if the HRegion is backed by a hdfs snapshot. I have a patch for trunk with some basic tests if folks are interested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14801) Enhance the Spark-HBase connector catalog with json format
[ https://issues.apache.org/jira/browse/HBASE-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-14801: --- Attachment: HBASE-14801-4.patch > Enhance the Spark-HBase connector catalog with json format > -- > > Key: HBASE-14801 > URL: https://issues.apache.org/jira/browse/HBASE-14801 > Project: HBase > Issue Type: Improvement >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Attachments: HBASE-14801-1.patch, HBASE-14801-2.patch, > HBASE-14801-3.patch, HBASE-14801-4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14801) Enhance the Spark-HBase connector catalog with json format
[
https://issues.apache.org/jira/browse/HBASE-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166383#comment-15166383
]
Hadoop QA commented on HBASE-14801:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 36s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 28s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 0s
{color} | {color:red} hbase-spark in master has 66 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 40s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 1m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 31s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 1m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
35m 41s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 15s
{color} | {color:red} hbase-spark introduced 7 new FindBugs issues. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 38s
{color} | {color:red} hbase-spark-jdk1.8.0_72 with JDK v1.8.0_72 generated 1
new issues (was 15, now 16). {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 2m 0s
{color} | {color:red} hbase-spark-jdk1.7.0_95 with JDK v1.7.0_95 generated 1
new issues (was 15, now 16). {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color
[jira] [Commented] (HBASE-14801) Enhance the Spark-HBase connector catalog with json format
[ https://issues.apache.org/jira/browse/HBASE-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166381#comment-15166381 ] Ted Yu commented on HBASE-14801: Zhan: The patch downloaded from review board is HBASE-14801-1.patch Can you upload the latest patch there ? Thanks > Enhance the Spark-HBase connector catalog with json format > -- > > Key: HBASE-14801 > URL: https://issues.apache.org/jira/browse/HBASE-14801 > Project: HBase > Issue Type: Improvement >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Attachments: HBASE-14801-1.patch, HBASE-14801-2.patch, > HBASE-14801-3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15187) Integrate CSRF prevention filter to REST gateway
[ https://issues.apache.org/jira/browse/HBASE-15187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15187: --- Attachment: HBASE-15187.v9.patch > Integrate CSRF prevention filter to REST gateway > > > Key: HBASE-15187 > URL: https://issues.apache.org/jira/browse/HBASE-15187 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: HBASE-15187.v1.patch, HBASE-15187.v2.patch, > HBASE-15187.v3.patch, HBASE-15187.v4.patch, HBASE-15187.v5.patch, > HBASE-15187.v6.patch, HBASE-15187.v7.patch, HBASE-15187.v8.patch, > HBASE-15187.v9.patch > > > HADOOP-12691 introduced a filter in Hadoop Common to help REST APIs guard > against cross-site request forgery attacks. > This issue tracks the integration of that filter into HBase REST gateway. > From REST section of refguide: > To delete a table, use a DELETE request with the /schema endpoint: > http://example.com:8000/schema > Suppose an attacker hosts a malicious web form on a domain under his control. > The form uses the DELETE action targeting a REST URL. Through social > engineering, the attacker tricks an authenticated user into accessing the > form and submitting it. > The browser sends the HTTP DELETE request to the REST gateway. > At REST gateway, the call is executed and user table is dropped -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15187) Integrate CSRF prevention filter to REST gateway
[ https://issues.apache.org/jira/browse/HBASE-15187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15187: --- Status: Patch Available (was: Open) > Integrate CSRF prevention filter to REST gateway > > > Key: HBASE-15187 > URL: https://issues.apache.org/jira/browse/HBASE-15187 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: HBASE-15187.v1.patch, HBASE-15187.v2.patch, > HBASE-15187.v3.patch, HBASE-15187.v4.patch, HBASE-15187.v5.patch, > HBASE-15187.v6.patch, HBASE-15187.v7.patch, HBASE-15187.v8.patch, > HBASE-15187.v9.patch > > > HADOOP-12691 introduced a filter in Hadoop Common to help REST APIs guard > against cross-site request forgery attacks. > This issue tracks the integration of that filter into HBase REST gateway. > From REST section of refguide: > To delete a table, use a DELETE request with the /schema endpoint: > http://example.com:8000/schema > Suppose an attacker hosts a malicious web form on a domain under his control. > The form uses the DELETE action targeting a REST URL. Through social > engineering, the attacker tricks an authenticated user into accessing the > form and submitting it. > The browser sends the HTTP DELETE request to the REST gateway. > At REST gateway, the call is executed and user table is dropped -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-10123) Change default ports; move them out of linux ephemeral port range
[ https://issues.apache.org/jira/browse/HBASE-10123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166328#comment-15166328 ] Jonathan Hsieh commented on HBASE-10123: Looks like in HDFS-9427 there is a motion to move all the hadoop ports in the 9xxx range. I've suggested there that hadoop-kms port be moved into that range as well. > Change default ports; move them out of linux ephemeral port range > - > > Key: HBASE-10123 > URL: https://issues.apache.org/jira/browse/HBASE-10123 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.1.1 >Reporter: stack >Assignee: Jonathan Hsieh >Priority: Critical > Fix For: 0.99.0 > > Attachments: hbase-10123.patch, hbase-10123.v2.patch, > hbase-10123.v3.patch, hbase-10123.v4.patch > > > Our defaults clash w/ the range linux assigns itself for creating come-and-go > ephemeral ports; likely in our history we've clashed w/ a random, short-lived > process. While easy to change the defaults, we should just ship w/ defaults > that make sense. We could host ourselves up into the 7 or 8k range. > See http://www.ncftp.com/ncftpd/doc/misc/ephemeral_ports.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14801) Enhance the Spark-HBase connector catalog with json format
[
https://issues.apache.org/jira/browse/HBASE-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166313#comment-15166313
]
Hadoop QA commented on HBASE-14801:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 42s
{color} | {color:red} hbase-spark in master has 66 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
29s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
2s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
24m 23s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 52s
{color} | {color:red} hbase-spark introduced 7 new FindBugs issues. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 2s
{color} | {color:red} hbase-spark-jdk1.8.0_72 with JDK v1.8.0_72 generated 1
new issues (was 15, now 16). {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 16s
{color} | {color:red} hbase-spark-jdk1.7.0_95 with JDK v1.7.0_95 generated 1
new issues (was 15, now 16). {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {
[jira] [Commented] (HBASE-15320) HBase connector for Kafka Connect
[ https://issues.apache.org/jira/browse/HBASE-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15165694#comment-15165694 ] Andrew Purtell commented on HBASE-15320: Sure. Updated description text. Thanks. > HBase connector for Kafka Connect > - > > Key: HBASE-15320 > URL: https://issues.apache.org/jira/browse/HBASE-15320 > Project: HBase > Issue Type: New Feature >Reporter: Andrew Purtell > Fix For: 2.0.0 > > > Implement an HBase connector with source and sink tasks for the Connect > framework (http://docs.confluent.io/2.0.0/connect/index.html) available in > Kafka 0.9 and later. > See also: > http://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines > An HBase source > (http://docs.confluent.io/2.0.0/connect/devguide.html#task-example-source-task) > could be implemented as a replication endpoint or WALObserver, publishing > cluster wide change streams from the WAL to one or more topics, with > configurable mapping and partitioning of table changes to topics. > An HBase sink task > (http://docs.confluent.io/2.0.0/connect/devguide.html#sink-tasks) would > persist, with optional transformation (JSON? Avro?, map fields to native > schema?), Kafka SinkRecords into HBase tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
