[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466966#comment-16466966 ] Hadoop QA commented on HBASE-20411: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 6 new or modified test files. {color} | || || || || {color:brown} branch-2.0 Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 40s{color} | {color:green} branch-2.0 passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 41s{color} | {color:green} branch-2.0 passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 18s{color} | {color:green} branch-2.0 passed {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m 1s{color} | {color:green} branch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 54s{color} | {color:green} branch-2.0 passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s{color} | {color:green} branch-2.0 passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 31s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 42s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 42s{color} | {color:red} hbase-server generated 6 new + 182 unchanged - 6 fixed = 188 total (was 188) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 15s{color} | {color:red} hbase-server: The patch generated 4 new + 493 unchanged - 5 fixed = 497 total (was 498) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m 8s{color} | {color:green} patch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 11m 0s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s{color} | {color:green} hbase-server generated 0 new + 0 unchanged - 2 fixed = 0 total (was 2) {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}110m 14s{color} | {color:red} hbase-server in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 21s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}148m 36s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush | | | hadoop.hbase.regionserver.TestHRegion | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:369877d | | JIRA Issue | HBASE-20411 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12922392/HBASE-20411.branch-2.0.011.patch | | Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 08324175daaf 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | branch-2.0 / 9653a4d0df | | maven | version: Apache Maven 3.5.3 (3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) | | Default Java | 1.8.0_171 | | findbugs | v3.1.0-RC3 | | javac | https://builds.apache.org/job/PreCommit-HBASE
[jira] [Assigned] (HBASE-6970) hbase-deamon.sh creates/updates pid file even when that start failed.
[ https://issues.apache.org/jira/browse/HBASE-6970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi reassigned HBASE-6970: Assignee: (was: Ashish Singhi) > hbase-deamon.sh creates/updates pid file even when that start failed. > - > > Key: HBASE-6970 > URL: https://issues.apache.org/jira/browse/HBASE-6970 > Project: HBase > Issue Type: Bug > Components: Usability >Reporter: Lars Hofhansl >Priority: Major > > We just ran into a strange issue where could neither start nor stop services > with hbase-deamon.sh. > The problem is this: > {code} > nohup nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase \ > --config "${HBASE_CONF_DIR}" \ > $command "$@" $startStop > "$logout" 2>&1 < /dev/null & > echo $! > $pid > {code} > So the pid file is created or updated even when the start of the service > failed. The next stop command will then fail, because the pid file has the > wrong pid in it. > Edit: Spelling and more spelling errors. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
[ https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466962#comment-16466962 ] Hadoop QA commented on HBASE-19369: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s{color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color} | {color:red} HBASE-19369 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | HBASE-19369 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12903918/HBASE-19369.v8.patch | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/12742/console | | Powered by | Apache Yetus 0.7.0 http://yetus.apache.org | This message was automatically generated. > HBase Should use Builder Pattern to Create Log Files while using WAL on > Erasure Coding > -- > > Key: HBASE-19369 > URL: https://issues.apache.org/jira/browse/HBASE-19369 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Alex Leblang >Assignee: Alex Leblang >Priority: Major > Attachments: HBASE-19369.master.001.patch, > HBASE-19369.master.002.patch, HBASE-19369.master.003.patch, > HBASE-19369.master.004.patch, HBASE-19369.v5.patch, HBASE-19369.v6.patch, > HBASE-19369.v7.patch, HBASE-19369.v8.patch > > > Right now an HBase instance using the WAL won't function properly in an > Erasure Coded environment. We should change the following line to use the > hdfs.DistributedFileSystem builder pattern > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20540) [umbrella] Hadoop 3 compatibility
[ https://issues.apache.org/jira/browse/HBASE-20540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466961#comment-16466961 ] Duo Zhang commented on HBASE-20540: --- [~elserj] FYI. > [umbrella] Hadoop 3 compatibility > - > > Key: HBASE-20540 > URL: https://issues.apache.org/jira/browse/HBASE-20540 > Project: HBase > Issue Type: Umbrella >Reporter: Duo Zhang >Priority: Major > Fix For: 2.1.0, 2.0.1 > > > There are known issues about the hadoop 3 compatibility for hbase 2. But > hadoop 3 is still not production ready. So we will link the issues here and > once there is a production ready hadoop 3 release, we will fix these issues > soon and upgrade our dependencies on hadoop, and also update the support > matrix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20502) Document HBase incompatible with Yarn 2.9.0 and 3.0.x due to YARN-7190
[ https://issues.apache.org/jira/browse/HBASE-20502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20502: -- Issue Type: Sub-task (was: Bug) Parent: HBASE-20540 > Document HBase incompatible with Yarn 2.9.0 and 3.0.x due to YARN-7190 > -- > > Key: HBASE-20502 > URL: https://issues.apache.org/jira/browse/HBASE-20502 > Project: HBase > Issue Type: Sub-task > Components: dependencies, documentation >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Blocker > Fix For: 3.0.0 > > Attachments: HBASE-20502.patch > > > We need to call out hadoop-yarn 2.9.0 and the entire 3.0.x line as explicitly > unsupported due to needing YARN-7190 fixed in versions that have ATS > available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
[ https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19369: -- Issue Type: Sub-task (was: Bug) Parent: HBASE-20540 > HBase Should use Builder Pattern to Create Log Files while using WAL on > Erasure Coding > -- > > Key: HBASE-19369 > URL: https://issues.apache.org/jira/browse/HBASE-19369 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Alex Leblang >Assignee: Alex Leblang >Priority: Major > Attachments: HBASE-19369.master.001.patch, > HBASE-19369.master.002.patch, HBASE-19369.master.003.patch, > HBASE-19369.master.004.patch, HBASE-19369.v5.patch, HBASE-19369.v6.patch, > HBASE-19369.v7.patch, HBASE-19369.v8.patch > > > Right now an HBase instance using the WAL won't function properly in an > Erasure Coded environment. We should change the following line to use the > hdfs.DistributedFileSystem builder pattern > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[ https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20244: -- Issue Type: Sub-task (was: Bug) Parent: HBASE-20540 > NoSuchMethodException when retrieving private method > decryptEncryptedDataEncryptionKey from DFSClient > - > > Key: HBASE-20244 > URL: https://issues.apache.org/jira/browse/HBASE-20244 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 2.0.1 > > Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt > > > I was running unit test against hadoop 3.0.1 RC and saw the following in test > output: > {code} > ERROR [RS-EventLoopGroup-3-3] > asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly > initialize access to HDFS internals. Please update your WAL Provider to not > make use of the 'asyncfs' provider. See HBASE-16110 for more information. > java.lang.NoSuchMethodException: > org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) > at java.lang.Class.getDeclaredMethod(Class.java:2130) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104) > at > org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) > {code} > The private method was moved by HDFS-12574 to HdfsKMSUtil with different > signature. > To accommodate the above method movement, it seems we need to call the > following method of DFSClient : > {code} > public KeyProvider getKeyProvider() throws IOException { > {code} > Since the new decryptEncryptedDataEncryptionKey method has this signature: > {code} > static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo > feInfo, KeyProvider keyProvider) throws IOException { > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20540) [umbrella] Hadoop 3 compatibility
Duo Zhang created HBASE-20540: - Summary: [umbrella] Hadoop 3 compatibility Key: HBASE-20540 URL: https://issues.apache.org/jira/browse/HBASE-20540 Project: HBase Issue Type: Umbrella Reporter: Duo Zhang Fix For: 2.1.0, 2.0.1 There are known issues about the hadoop 3 compatibility for hbase 2. But hadoop 3 is still not production ready. So we will link the issues here and once there is a production ready hadoop 3 release, we will fix these issues soon and upgrade our dependencies on hadoop, and also update the support matrix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19064) Synchronous replication for HBase
[ https://issues.apache.org/jira/browse/HBASE-19064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466945#comment-16466945 ] Hudson commented on HBASE-19064: Results for branch HBASE-19064 [build #122 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/122/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/122//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/122//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/122//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > Synchronous replication for HBase > - > > Key: HBASE-19064 > URL: https://issues.apache.org/jira/browse/HBASE-19064 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0 > > > The guys from Alibaba made a presentation on HBaseCon Asia about the > synchronous replication for HBase. We(Xiaomi) think this is a very useful > feature for HBase so we want to bring it into the community version. > This is a big feature so we plan to do it in a feature branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20500) [rsgroup] should keep at least one server in default group
[ https://issues.apache.org/jira/browse/HBASE-20500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466941#comment-16466941 ] Yechao Chen commented on HBASE-20500: - sorry for reply later, thank you for your review and suggestion,[~yuzhih...@gmail.com] change the code by your suggestion, TestShellRSGroups has passed locally. submit v5 and retry qa > [rsgroup] should keep at least one server in default group > -- > > Key: HBASE-20500 > URL: https://issues.apache.org/jira/browse/HBASE-20500 > Project: HBase > Issue Type: Bug > Components: rsgroup >Affects Versions: 2.0.0 >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Major > Attachments: HBASE-20500-branch-2.v1.patch, > HBASE-20500-master.v1.patch, HBASE-20500-master.v2.patch, > HBASE-20500-master.v3.patch, HBASE-20500-master.v4.patch, > HBASE-20500-master.v5.patch > > > we move all the servers from default group > the default group will has no servers, > then we create a new table , > it will failed case of the default group has no servers > eorr info is : > EROOR: ConstraintException:Target RSGroup must have at lease on server > > we should keep at least one server in 'default' RSGroup > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20500) [rsgroup] should keep at least one server in default group
[ https://issues.apache.org/jira/browse/HBASE-20500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-20500: Status: Patch Available (was: Open) > [rsgroup] should keep at least one server in default group > -- > > Key: HBASE-20500 > URL: https://issues.apache.org/jira/browse/HBASE-20500 > Project: HBase > Issue Type: Bug > Components: rsgroup >Affects Versions: 2.0.0 >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Major > Attachments: HBASE-20500-branch-2.v1.patch, > HBASE-20500-master.v1.patch, HBASE-20500-master.v2.patch, > HBASE-20500-master.v3.patch, HBASE-20500-master.v4.patch, > HBASE-20500-master.v5.patch > > > we move all the servers from default group > the default group will has no servers, > then we create a new table , > it will failed case of the default group has no servers > eorr info is : > EROOR: ConstraintException:Target RSGroup must have at lease on server > > we should keep at least one server in 'default' RSGroup > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20500) [rsgroup] should keep at least one server in default group
[ https://issues.apache.org/jira/browse/HBASE-20500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-20500: Attachment: HBASE-20500-master.v5.patch > [rsgroup] should keep at least one server in default group > -- > > Key: HBASE-20500 > URL: https://issues.apache.org/jira/browse/HBASE-20500 > Project: HBase > Issue Type: Bug > Components: rsgroup >Affects Versions: 2.0.0 >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Major > Attachments: HBASE-20500-branch-2.v1.patch, > HBASE-20500-master.v1.patch, HBASE-20500-master.v2.patch, > HBASE-20500-master.v3.patch, HBASE-20500-master.v4.patch, > HBASE-20500-master.v5.patch > > > we move all the servers from default group > the default group will has no servers, > then we create a new table , > it will failed case of the default group has no servers > eorr info is : > EROOR: ConstraintException:Target RSGroup must have at lease on server > > we should keep at least one server in 'default' RSGroup > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466925#comment-16466925 ] Hadoop QA commented on HBASE-20004: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 59s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 34s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 59s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m 2s{color} | {color:green} branch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 18s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 48s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 48s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 9s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m 30s{color} | {color:green} patch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 17m 33s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 53s{color} | {color:green} hbase-http in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 8s{color} | {color:green} hbase-thrift in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 51s{color} | {color:green} hbase-rest in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 30s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 60m 13s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f | | JIRA Issue | HBASE-20004 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12922263/HBASE-20004.v1.patch | | Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d11076ca0f79 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@
[jira] [Assigned] (HBASE-5622) Improve efficiency of mapred vesion of RowCounter
[ https://issues.apache.org/jira/browse/HBASE-5622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi reassigned HBASE-5622: Assignee: (was: Ashish Singhi) > Improve efficiency of mapred vesion of RowCounter > - > > Key: HBASE-5622 > URL: https://issues.apache.org/jira/browse/HBASE-5622 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Priority: Minor > Attachments: HBASE-5622.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-5622) Improve efficiency of mapred vesion of RowCounter
[ https://issues.apache.org/jira/browse/HBASE-5622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-5622: - Status: Open (was: Patch Available) > Improve efficiency of mapred vesion of RowCounter > - > > Key: HBASE-5622 > URL: https://issues.apache.org/jira/browse/HBASE-5622 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Priority: Minor > Attachments: HBASE-5622.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-5742) LoadIncrementalHFiles throws too generic of an exception
[ https://issues.apache.org/jira/browse/HBASE-5742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi resolved HBASE-5742. -- Resolution: Not A Problem The constructor in the context no more exists. > LoadIncrementalHFiles throws too generic of an exception > > > Key: HBASE-5742 > URL: https://issues.apache.org/jira/browse/HBASE-5742 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: David Capwell >Assignee: Ashish Singhi >Priority: Major > > In 0.90 the LoadIncrementalHFiles constructor did not throw an exception, now > it throws Exception. The constructor should ether not throw an exception or > throw ZooKeeperConnectionException, and MasterNotRunningException since those > come from the HBaseAdmin call. > https://github.com/apache/hbase/blob/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java#L105 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-13644) Expand AC testing coverage to include all combinations of scope and permissions for region interface
[ https://issues.apache.org/jira/browse/HBASE-13644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-13644: -- Labels: beginner (was: ) > Expand AC testing coverage to include all combinations of scope and > permissions for region interface > > > Key: HBASE-13644 > URL: https://issues.apache.org/jira/browse/HBASE-13644 > Project: HBase > Issue Type: Sub-task > Components: security, test >Reporter: Ashish Singhi >Priority: Major > Labels: beginner > > As of now, the tests in TestAccessController and TestAccessController2 > doesn't cover all the combinations of Scope and Permissions. Ideally, we > should have testing coverage for the entire region interface in [ACL > matrix|https://hbase.apache.org/book/appendix_acl_matrix.html]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-13644) Expand AC testing coverage to include all combinations of scope and permissions for region interface
[ https://issues.apache.org/jira/browse/HBASE-13644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi reassigned HBASE-13644: - Assignee: (was: Ashish Singhi) > Expand AC testing coverage to include all combinations of scope and > permissions for region interface > > > Key: HBASE-13644 > URL: https://issues.apache.org/jira/browse/HBASE-13644 > Project: HBase > Issue Type: Sub-task > Components: security, test >Reporter: Ashish Singhi >Priority: Major > > As of now, the tests in TestAccessController and TestAccessController2 > doesn't cover all the combinations of Scope and Permissions. Ideally, we > should have testing coverage for the entire region interface in [ACL > matrix|https://hbase.apache.org/book/appendix_acl_matrix.html]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466891#comment-16466891 ] stack commented on HBASE-20411: --- Review appreciated. Thanks. > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > HBASE-20411.branch-2.0.011.patch, jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20539) Disable IMC; part 2
stack created HBASE-20539: - Summary: Disable IMC; part 2 Key: HBASE-20539 URL: https://issues.apache.org/jira/browse/HBASE-20539 Project: HBase Issue Type: Sub-task Components: in-memory-compaction Reporter: stack Assignee: stack Fix For: 2.0.1 Just noticed that post-flush, we are picking up a CompactingMemStore though IMC is supposed to be off. We are picking up an inMemoryCompaction of BASIC (noticed a day or so ago by [~chia7712]) and so, we fall into the default in HStore#getMemstore which is set to be a CompactingMemStore No real perf difference going by the compares I've been running but let me clean this up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20500) [rsgroup] should keep at least one server in default group
[ https://issues.apache.org/jira/browse/HBASE-20500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-20500: Affects Version/s: (was: 2.0.0-beta-2) 2.0.0 > [rsgroup] should keep at least one server in default group > -- > > Key: HBASE-20500 > URL: https://issues.apache.org/jira/browse/HBASE-20500 > Project: HBase > Issue Type: Bug > Components: rsgroup >Affects Versions: 2.0.0 >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Major > Attachments: HBASE-20500-branch-2.v1.patch, > HBASE-20500-master.v1.patch, HBASE-20500-master.v2.patch, > HBASE-20500-master.v3.patch, HBASE-20500-master.v4.patch > > > we move all the servers from default group > the default group will has no servers, > then we create a new table , > it will failed case of the default group has no servers > eorr info is : > EROOR: ConstraintException:Target RSGroup must have at lease on server > > we should keep at least one server in 'default' RSGroup > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[ https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466867#comment-16466867 ] Duo Zhang commented on HBASE-20244: --- Let’s open a umbrella issue for Hadoop 3 compatibility? And linked the known issues there. Once the production ready Hadoop 3 is out, let’s fix these things ASAP and release a new 2.0.x. > NoSuchMethodException when retrieving private method > decryptEncryptedDataEncryptionKey from DFSClient > - > > Key: HBASE-20244 > URL: https://issues.apache.org/jira/browse/HBASE-20244 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 2.0.1 > > Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt > > > I was running unit test against hadoop 3.0.1 RC and saw the following in test > output: > {code} > ERROR [RS-EventLoopGroup-3-3] > asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly > initialize access to HDFS internals. Please update your WAL Provider to not > make use of the 'asyncfs' provider. See HBASE-16110 for more information. > java.lang.NoSuchMethodException: > org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) > at java.lang.Class.getDeclaredMethod(Class.java:2130) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104) > at > org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) > {code} > The private method was moved by HDFS-12574 to HdfsKMSUtil with different > signature. > To accommodate the above method movement, it seems we need to call the > following method of DFSClient : > {code} > public KeyProvider getKeyProvider() throws IOException { > {code} > Since the new decryptEncryptedDataEncryptionKey method has this signature: > {code} > static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo > feInfo, KeyProvider keyProvider) throws IOException { > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[ https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466863#comment-16466863 ] Josh Elser commented on HBASE-20244: Wow. Ok. A simple "Yes, I disagree with you" would've been more than sufficient. Apparently this is a touchy subject. Just so you know, I will be revisiting this when a "production ready" Hadoop 3 version is out. This is slated to be 3.1.1 or 3.1.2. This can be kicked down the road, but not very far. > NoSuchMethodException when retrieving private method > decryptEncryptedDataEncryptionKey from DFSClient > - > > Key: HBASE-20244 > URL: https://issues.apache.org/jira/browse/HBASE-20244 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 2.0.1 > > Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt > > > I was running unit test against hadoop 3.0.1 RC and saw the following in test > output: > {code} > ERROR [RS-EventLoopGroup-3-3] > asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly > initialize access to HDFS internals. Please update your WAL Provider to not > make use of the 'asyncfs' provider. See HBASE-16110 for more information. > java.lang.NoSuchMethodException: > org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) > at java.lang.Class.getDeclaredMethod(Class.java:2130) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104) > at > org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) > {code} > The private method was moved by HDFS-12574 to HdfsKMSUtil with different > signature. > To accommodate the above method movement, it seems we need to call the > following method of DFSClient : > {code} > public KeyProvider getKeyProvider() throws IOException { > {code} > Since the new decryptEncryptedDataEncryptionKey method has this signature: > {code} > static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo > feInfo, KeyProvider keyProvider) throws IOException { > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-20004: -- Status: Patch Available (was: In Progress) > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Attachments: HBASE-20004.branch-1.patch, HBASE-20004.patch, > HBASE-20004.v1.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
[ https://issues.apache.org/jira/browse/HBASE-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466851#comment-16466851 ] Hudson commented on HBASE-20536: Results for branch branch-2 [build #708 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/708/]: (/) *{color:green}+1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/708//General_Nightly_Build_Report/] (/) {color:green}+1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/708//JDK8_Nightly_Build_Report_(Hadoop2)/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/708//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > Make TestRegionServerAccounting stable and it should not use absolute number > > > Key: HBASE-20536 > URL: https://issues.apache.org/jira/browse/HBASE-20536 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20536.master.001.patch > > > TestRegionServerAccounting failed on our internal jenkin job as we config Xmx > to 10G. We should modify the absolute number to relative value. > {code:java} > new MemStoreSize((3L * 1024L * 1024L * 1024L), (1L * 1024L * 1024L * 1024L), > 0);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20411: -- Attachment: HBASE-20411.branch-2.0.011.patch > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > HBASE-20411.branch-2.0.011.patch, jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466830#comment-16466830 ] stack commented on HBASE-20411: --- Running on cluster with this patch, there is before [^2.pe.write.32026.lock.svg] , where the only lock that shows is the MutableSegment increment lock and then after, [^2.pe.write.ameliorate.106553.lock.svg] , where we have a bunch of different locks showing up. Here is how this MutableSegment lock shows in !jmc6.write_time_locks.png! > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20411: -- Attachment: 2.pe.write.ameliorate.106553.lock.svg > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20411: -- Attachment: 2.pe.write.32026.lock.svg > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20411: -- Attachment: jmc6.write_time_locks.png > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, 41901.lock.svg, > HBASE-20411-atomiclong-longadder.patch, HBASE-20411.branch-2.0.001.patch, > HBASE-20411.branch-2.0.002.patch, HBASE-20411.branch-2.0.003.patch, > HBASE-20411.branch-2.0.004.patch, HBASE-20411.branch-2.0.005.patch, > HBASE-20411.branch-2.0.006.patch, HBASE-20411.branch-2.0.007.patch, > HBASE-20411.branch-2.0.008.patch, HBASE-20411.branch-2.0.009.patch, > HBASE-20411.branch-2.0.010.patch, jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466786#comment-16466786 ] Hudson commented on HBASE-20505: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1110 (See [https://builds.apache.org/job/HBase-1.2-IT/1110/]) HBASE-20505 PE should support multi column family read and write cases (apurtell: rev ef9fd29cad44437d3db185931c50598621bc125a) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java * (edit) hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestRegionReplicaPerf.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20535) Check the UT TestSaslFanOutOneBlockAsyncDFSOutput which is always flaky
[ https://issues.apache.org/jira/browse/HBASE-20535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu resolved HBASE-20535. -- Resolution: Duplicate Close this, because [~stack] had handled it. > Check the UT TestSaslFanOutOneBlockAsyncDFSOutput which is always flaky > --- > > Key: HBASE-20535 > URL: https://issues.apache.org/jira/browse/HBASE-20535 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > > 1. > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html > 2. https://builds.apache.org/job/HBASE-Flaky-Tests//30949 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466780#comment-16466780 ] Hudson commented on HBASE-20505: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #403 (See [https://builds.apache.org/job/HBase-1.3-IT/403/]) HBASE-20505 PE should support multi column family read and write cases (apurtell: rev d5f3938520ecef6b2369ba762296b18f7266b1a7) * (edit) hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestRegionReplicaPerf.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[ https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466779#comment-16466779 ] Duo Zhang commented on HBASE-20244: --- So I suggest we add this into our ref guide: HBase-2.0.0 has been tested with hadoop-2.7.x, and hadoop-3.0.0. The hbase binary is shipped with hadoop-2.7.4, but it is also OK to run this binary on HDFS 2.8+, including 3.0.x and 3.1.x. If you want to HBase to use the dfs client other than 2.7.4, for example, you want HBase to write file with EC, you need to build HBase by yourself with other versions of hadoop. Notice that the default config for HBase can only work with hadoop-3.0.0, if you want to use other versions please set the 'hbase.wal.provider' to 'filesystem' in hbase-site.xml. And please do not enable EC for the WAL directory. Hadoop3 is not production ready so the support is not perfect, bug report is appreciated. > NoSuchMethodException when retrieving private method > decryptEncryptedDataEncryptionKey from DFSClient > - > > Key: HBASE-20244 > URL: https://issues.apache.org/jira/browse/HBASE-20244 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 2.0.1 > > Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt > > > I was running unit test against hadoop 3.0.1 RC and saw the following in test > output: > {code} > ERROR [RS-EventLoopGroup-3-3] > asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly > initialize access to HDFS internals. Please update your WAL Provider to not > make use of the 'asyncfs' provider. See HBASE-16110 for more information. > java.lang.NoSuchMethodException: > org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) > at java.lang.Class.getDeclaredMethod(Class.java:2130) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104) > at > org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) > {code} > The private method was moved by HDFS-12574 to HdfsKMSUtil with different > signature. > To accommodate the above method movement, it seems we need to call the > following method of DFSClient : > {code} > public KeyProvider getKeyProvider() throws IOException { > {code} > Since the new decryptEncryptedDataEncryptionKey method has this signature: > {code} > static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo > feInfo, KeyProvider keyProvider) throws IOException { > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-20505. Resolution: Fixed Pushed to 1.2 and up > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[ https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466752#comment-16466752 ] Duo Zhang edited comment on HBASE-20244 at 5/8/18 2:27 AM: --- And there are also other problems for WAL on hadoop3, not only this one. If user sets EC on the WAL directory, then we will fail to write WAL because the EC output does not support hflush/hsync. We need to use an API which only introduced in Hadoop 3 to create a normal output stream. This also effect FSHLog, not only AsyncFSWAL. In general, before we have a production ready hadoop3 release, I do not think we need to make the Hadoop3 support perfect. was (Author: apache9): And there are also other problems for WAL on hadoop3, not only this one. If user sets EC on the WAL directory, then we will fail to write WAL because the EC output does not support hflush/haunches. We need to use an API which only introduced in Hadoop 3 to create a normal output stream. This also effect FSHLog, not only AsyncFSWAL. In general, before we have a production ready hadoop3 release, I do not think we need to make the Hadoop3 support perfect. > NoSuchMethodException when retrieving private method > decryptEncryptedDataEncryptionKey from DFSClient > - > > Key: HBASE-20244 > URL: https://issues.apache.org/jira/browse/HBASE-20244 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 2.0.1 > > Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt > > > I was running unit test against hadoop 3.0.1 RC and saw the following in test > output: > {code} > ERROR [RS-EventLoopGroup-3-3] > asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly > initialize access to HDFS internals. Please update your WAL Provider to not > make use of the 'asyncfs' provider. See HBASE-16110 for more information. > java.lang.NoSuchMethodException: > org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) > at java.lang.Class.getDeclaredMethod(Class.java:2130) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104) > at > org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) > {code} > The private method was moved by HDFS-12574 to HdfsKMSUtil with different > signature. > To accommodate the above method movement, it seems we need to call the > following method of DFSClient : > {code} > public KeyProvider getKeyProvider() throws IOException { > {code} > Since the new decryptEncryptedDataEncryptionKey method has this signature: > {code} > static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo > feInfo, KeyProvider keyProvider) throws IOException { > {code} --
[jira] [Commented] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[ https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466752#comment-16466752 ] Duo Zhang commented on HBASE-20244: --- And there are also other problems for WAL on hadoop3, not only this one. If user sets EC on the WAL directory, then we will fail to write WAL because the EC output does not support hflush/haunches. We need to use an API which only introduced in Hadoop 3 to create a normal output stream. This also effect FSHLog, not only AsyncFSWAL. In general, before we have a production ready hadoop3 release, I do not think we need to make the Hadoop3 support perfect. > NoSuchMethodException when retrieving private method > decryptEncryptedDataEncryptionKey from DFSClient > - > > Key: HBASE-20244 > URL: https://issues.apache.org/jira/browse/HBASE-20244 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 2.0.1 > > Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt > > > I was running unit test against hadoop 3.0.1 RC and saw the following in test > output: > {code} > ERROR [RS-EventLoopGroup-3-3] > asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly > initialize access to HDFS internals. Please update your WAL Provider to not > make use of the 'asyncfs' provider. See HBASE-16110 for more information. > java.lang.NoSuchMethodException: > org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) > at java.lang.Class.getDeclaredMethod(Class.java:2130) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104) > at > org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) > {code} > The private method was moved by HDFS-12574 to HdfsKMSUtil with different > signature. > To accommodate the above method movement, it seems we need to call the > following method of DFSClient : > {code} > public KeyProvider getKeyProvider() throws IOException { > {code} > Since the new decryptEncryptedDataEncryptionKey method has this signature: > {code} > static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo > feInfo, KeyProvider keyProvider) throws IOException { > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[ https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466747#comment-16466747 ] Duo Zhang commented on HBASE-20244: --- And if you really want to use Hadoop 3.0.1 or other hadoop3 versions, you can specify the WAL implementation in the configuration file by yourself. I stand my point that we do not need to change our configs for a not production ready version. The bug here is easy to fix, the patch has already been uploaded, but I just do not want to waste time here because finally we will not add a unstable version into our support matrix. > NoSuchMethodException when retrieving private method > decryptEncryptedDataEncryptionKey from DFSClient > - > > Key: HBASE-20244 > URL: https://issues.apache.org/jira/browse/HBASE-20244 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 2.0.1 > > Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt > > > I was running unit test against hadoop 3.0.1 RC and saw the following in test > output: > {code} > ERROR [RS-EventLoopGroup-3-3] > asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly > initialize access to HDFS internals. Please update your WAL Provider to not > make use of the 'asyncfs' provider. See HBASE-16110 for more information. > java.lang.NoSuchMethodException: > org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) > at java.lang.Class.getDeclaredMethod(Class.java:2130) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104) > at > org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) > {code} > The private method was moved by HDFS-12574 to HdfsKMSUtil with different > signature. > To accommodate the above method movement, it seems we need to call the > following method of DFSClient : > {code} > public KeyProvider getKeyProvider() throws IOException { > {code} > Since the new decryptEncryptedDataEncryptionKey method has this signature: > {code} > static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo > feInfo, KeyProvider keyProvider) throws IOException { > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[ https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466744#comment-16466744 ] Duo Zhang commented on HBASE-20244: --- You can use Hadoop 3.0.0. > NoSuchMethodException when retrieving private method > decryptEncryptedDataEncryptionKey from DFSClient > - > > Key: HBASE-20244 > URL: https://issues.apache.org/jira/browse/HBASE-20244 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 2.0.1 > > Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt > > > I was running unit test against hadoop 3.0.1 RC and saw the following in test > output: > {code} > ERROR [RS-EventLoopGroup-3-3] > asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly > initialize access to HDFS internals. Please update your WAL Provider to not > make use of the 'asyncfs' provider. See HBASE-16110 for more information. > java.lang.NoSuchMethodException: > org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) > at java.lang.Class.getDeclaredMethod(Class.java:2130) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104) > at > org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) > {code} > The private method was moved by HDFS-12574 to HdfsKMSUtil with different > signature. > To accommodate the above method movement, it seems we need to call the > following method of DFSClient : > {code} > public KeyProvider getKeyProvider() throws IOException { > {code} > Since the new decryptEncryptedDataEncryptionKey method has this signature: > {code} > static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo > feInfo, KeyProvider keyProvider) throws IOException { > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
[ https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466740#comment-16466740 ] Josh Elser commented on HBASE-20244: {quote}Users need to build it by themselves, so I think adding something in the refguide is enough? Especially that Hadoop 3 is not production ready. {quote} I think I just fundamentally disagree with you here. We expect our users to be trying out Hadoop3, regardless of whether it's production ready or not. That's not a justification for us to ship a known-broken feature as "on-by-default" in my book. Was trying to find a happy medium by suggesting that we change what the default WAL impl is for only Hadoop3 and not for Hadoop2... Can you tell me how this strikes you, please? > NoSuchMethodException when retrieving private method > decryptEncryptedDataEncryptionKey from DFSClient > - > > Key: HBASE-20244 > URL: https://issues.apache.org/jira/browse/HBASE-20244 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 2.0.1 > > Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt > > > I was running unit test against hadoop 3.0.1 RC and saw the following in test > output: > {code} > ERROR [RS-EventLoopGroup-3-3] > asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly > initialize access to HDFS internals. Please update your WAL Provider to not > make use of the 'asyncfs' provider. See HBASE-16110 for more information. > java.lang.NoSuchMethodException: > org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) > at java.lang.Class.getDeclaredMethod(Class.java:2130) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) > at > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104) > at > org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633) > at > org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) > {code} > The private method was moved by HDFS-12574 to HdfsKMSUtil with different > signature. > To accommodate the above method movement, it seems we need to call the > following method of DFSClient : > {code} > public KeyProvider getKeyProvider() throws IOException { > {code} > Since the new decryptEncryptedDataEncryptionKey method has this signature: > {code} > static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo > feInfo, KeyProvider keyProvider) throws IOException { > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20521) TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script run fail
[ https://issues.apache.org/jira/browse/HBASE-20521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466698#comment-16466698 ] Michael Jin commented on HBASE-20521: - Ted, Mike, thanks! > TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script > run fail > --- > > Key: HBASE-20521 > URL: https://issues.apache.org/jira/browse/HBASE-20521 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.0.0 > Environment: spark 2.2.1, hbase 2.0.0 >Reporter: Michael Jin >Assignee: Michael Jin >Priority: Major > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: HBASE-20521.master.001.patch, > HBASE-20521.master.002.patch > > > HBASE-20295 fix null point exception of "conf" member variable, add > "context.getConfiguration()" in case when "conf" object was not been properly > initialized, and put it into the first priority checking sequence, this code > change affect user call "setConf" explicitly initialize "conf" object in > TableOutputFormat object, proposal to change checking sequence, use "conf" > object from "getConf" method first . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
[ https://issues.apache.org/jira/browse/HBASE-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466690#comment-16466690 ] Guanghao Zhang commented on HBASE-20536: Pushed to master and branch-2. > Make TestRegionServerAccounting stable and it should not use absolute number > > > Key: HBASE-20536 > URL: https://issues.apache.org/jira/browse/HBASE-20536 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20536.master.001.patch > > > TestRegionServerAccounting failed on our internal jenkin job as we config Xmx > to 10G. We should modify the absolute number to relative value. > {code:java} > new MemStoreSize((3L * 1024L * 1024L * 1024L), (1L * 1024L * 1024L * 1024L), > 0);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
[ https://issues.apache.org/jira/browse/HBASE-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-20536: --- Resolution: Fixed Fix Version/s: 2.1.0 3.0.0 Status: Resolved (was: Patch Available) > Make TestRegionServerAccounting stable and it should not use absolute number > > > Key: HBASE-20536 > URL: https://issues.apache.org/jira/browse/HBASE-20536 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20536.master.001.patch > > > TestRegionServerAccounting failed on our internal jenkin job as we config Xmx > to 10G. We should modify the absolute number to relative value. > {code:java} > new MemStoreSize((3L * 1024L * 1024L * 1024L), (1L * 1024L * 1024L * 1024L), > 0);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19722) Implement a meta query statistics metrics source
[ https://issues.apache.org/jira/browse/HBASE-19722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466655#comment-16466655 ] Andrew Purtell commented on HBASE-19722: Thanks for the patch! We want this to be optional, but making it an example seems wrong. It should live in the hbase-server module. Please move it. Optional doesn't mean example. I think {{hermitMode}} is a fun name but probably better just to name this boolean something like "active"? Otherwise people might be a little confused when looking at the code in places. I don't think you need to test for existence when marking a meter. Ahead of marking you always register the meter if it doesn't exist. One interesting aspect not yet handled is removing all of the metrics from stop(). The meta region can move around. It will be closed on one server and reopened on another. Ideally we don't want stale metrics hanging around on the old server. If the metrics API doesn't support this we should add it. > That's why I prefer not to put topk calculation to this coprocessor since it > has a performance impact on all meta table operations. I do think we need to do the top-K in the coprocessor. Or, at least, prune meters that are not in the top K. Otherwise the amount of heap consumed by these metrics can grow rather large depending on the number of unique clients. > Could we count the types of access Ideally, we should have - Counts for each type of operation - Aggregate count of all accesses by client identity - Counts for each type of operation by client identity See my comment above about pruning meters and state for rare clients. The combinatorics of the above means we'd have 10 or more metrics per unique client. > Implement a meta query statistics metrics source > > > Key: HBASE-19722 > URL: https://issues.apache.org/jira/browse/HBASE-19722 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Xu Cang >Priority: Major > Attachments: HBASE-19722.patch, HBASE-19722.patch.1, > HBASE-19722.patch.2, HBASE-19722.patch.3, HBASE-19722.patch.4, > HBASE-19722.patch.5 > > > Implement a meta query statistics metrics source, created whenever a > regionserver starts hosting meta, removed when meta hosting moves. Provide > views on top tables by request counts, top meta rowkeys by request count, top > clients making requests by their hostname. > Can be implemented as a coprocessor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466605#comment-16466605 ] Andrew Purtell commented on HBASE-20505: Ok, committing shortly unless objection > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20259) Doc configs for in-memory-compaction and add detail to in-memory-compaction logging
[ https://issues.apache.org/jira/browse/HBASE-20259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20259: -- Release Note: Disables in-memory compaction as default (See HBASE-20464 for actually disabling IMC as default). Adds logging of in-memory compaction configuration on creation. Adds a chapter to the refguide on this new feature. was: Disables in-memory compaction as default. Adds logging of in-memory compaction configuration on creation. Adds a chapter to the refguide on this new feature. > Doc configs for in-memory-compaction and add detail to in-memory-compaction > logging > --- > > Key: HBASE-20259 > URL: https://issues.apache.org/jira/browse/HBASE-20259 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-20259.branch-2.0.ADDENDUM.patch, > HBASE-20259.master.001.patch, HBASE-20259.master.002.patch, > HBASE-20259.master.003.patch > > > I set {{hbase.systemtables.compacting.memstore.type}} to NONE but it seems > like in-memory is still on. My table looks like this: > {code} > Table ycsb is ENABLED > ycsb > COLUMN FAMILIES DESCRIPTION > {NAME => 'family', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false', > NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE', > CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => > 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER = > > 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMORY => 'false', > > CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', > > COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} > {code} > Looks like table doesn't have it on either (IN_MEMORY_COMPACTION doesn't show > in the above). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466584#comment-16466584 ] stack commented on HBASE-20505: --- Please [~apurtell] I've moved to PE for testing perf lately. This stuff and [~ram_krish]'s change help. > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466564#comment-16466564 ] Andrew Purtell commented on HBASE-20505: Rebased patches. Tested manually on a cluster. Used {{./bin/hbase pe --nomapred --rows=100 --families=10 randomWrite 1}} to write 1M rows with 10 families Verified expected 10 family schema with the shell. Then, {{./bin/hbase pe --nomapred --rows=100 --families=1 scanRange1000 1}} to read ranges of 1000 rows over one of the families. Then, {{./bin/hbase pe --nomapred --rows=100 --families=10 scanRange1000 1}} to read ranges of 1000 rows over all ten of the families. Bytes per result in scan metrics is the expected ~10x of the single family run. Mind if I put this in branch-2.0 along with the rest [~stack] ? > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20505: --- Fix Version/s: 1.4.5 2.0.1 1.3.3 > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20505: --- Attachment: (was: HBASE-20505-branch-1.patch) > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20505: --- Attachment: (was: HBASE-20505.patch) > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20505: --- Attachment: HBASE-20505.patch HBASE-20505-branch-1.patch > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466548#comment-16466548 ] Hudson commented on HBASE-20538: Results for branch branch-2.0 [build #271 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/271/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/271//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/271//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/271//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: > 0001-HBASE-20538-TestSaslFanOutOneBlockAsyncDFSOutput-DISABLE.patch > > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466537#comment-16466537 ] Hudson commented on HBASE-20523: Results for branch branch-1.2 [build #324 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/324/]: (x) *{color:red}-1 overall{color}* details (if available): (x) {color:red}-1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/324//General_Nightly_Build_Report/] (x) {color:red}-1 jdk7 checks{color} -- For more information [see jdk7 report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/324//JDK7_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/324//JDK8_Nightly_Build_Report_(Hadoop2)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 1.5.0, 1.2.7, 2.0.1, 1.4.5 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch, > HBASE-20523_branch-1.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20521) TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script run fail
[ https://issues.apache.org/jira/browse/HBASE-20521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466525#comment-16466525 ] Ted Yu commented on HBASE-20521: Mike - take your time, please merge tomorrow. > TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script > run fail > --- > > Key: HBASE-20521 > URL: https://issues.apache.org/jira/browse/HBASE-20521 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.0.0 > Environment: spark 2.2.1, hbase 2.0.0 >Reporter: Michael Jin >Assignee: Michael Jin >Priority: Major > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: HBASE-20521.master.001.patch, > HBASE-20521.master.002.patch > > > HBASE-20295 fix null point exception of "conf" member variable, add > "context.getConfiguration()" in case when "conf" object was not been properly > initialized, and put it into the first priority checking sequence, this code > change affect user call "setConf" explicitly initialize "conf" object in > TableOutputFormat object, proposal to change checking sequence, use "conf" > object from "getConf" method first . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466519#comment-16466519 ] Hudson commented on HBASE-20538: Results for branch branch-2 [build #707 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/707/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/707//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/707//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/707//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: > 0001-HBASE-20538-TestSaslFanOutOneBlockAsyncDFSOutput-DISABLE.patch > > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20521) TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script run fail
[ https://issues.apache.org/jira/browse/HBASE-20521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466515#comment-16466515 ] Mike Drob commented on HBASE-20521: --- minor nit: you lost a space between if and opening parenthesis in TableOutputFormat. I (or somebody else) can fix this on commit. I ran the patch and also tested it against my pig script and it looks like things work. Ran the unit test against a version that only uses getConf() and doesn't read the context conf and it failed as expected. So we're good on that side. [~tedyu] - if you have no further comments on RB feel free to push, otherwise I'll pick this up tomorrow. Would like to see it in master, branch-2, and branch-2.0 > TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script > run fail > --- > > Key: HBASE-20521 > URL: https://issues.apache.org/jira/browse/HBASE-20521 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.0.0 > Environment: spark 2.2.1, hbase 2.0.0 >Reporter: Michael Jin >Assignee: Michael Jin >Priority: Major > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: HBASE-20521.master.001.patch, > HBASE-20521.master.002.patch > > > HBASE-20295 fix null point exception of "conf" member variable, add > "context.getConfiguration()" in case when "conf" object was not been properly > initialized, and put it into the first priority checking sequence, this code > change affect user call "setConf" explicitly initialize "conf" object in > TableOutputFormat object, proposal to change checking sequence, use "conf" > object from "getConf" method first . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20411: -- Attachment: HBASE-20411.branch-2.0.010.patch > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, 41901.lock.svg, > HBASE-20411-atomiclong-longadder.patch, HBASE-20411.branch-2.0.001.patch, > HBASE-20411.branch-2.0.002.patch, HBASE-20411.branch-2.0.003.patch, > HBASE-20411.branch-2.0.004.patch, HBASE-20411.branch-2.0.005.patch, > HBASE-20411.branch-2.0.006.patch, HBASE-20411.branch-2.0.007.patch, > HBASE-20411.branch-2.0.008.patch, HBASE-20411.branch-2.0.009.patch, > HBASE-20411.branch-2.0.010.patch > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20411: -- Attachment: HBASE-20411.branch-2.0.009.patch > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, 41901.lock.svg, > HBASE-20411-atomiclong-longadder.patch, HBASE-20411.branch-2.0.001.patch, > HBASE-20411.branch-2.0.002.patch, HBASE-20411.branch-2.0.003.patch, > HBASE-20411.branch-2.0.004.patch, HBASE-20411.branch-2.0.005.patch, > HBASE-20411.branch-2.0.006.patch, HBASE-20411.branch-2.0.007.patch, > HBASE-20411.branch-2.0.008.patch, HBASE-20411.branch-2.0.009.patch, > HBASE-20411.branch-2.0.010.patch > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466454#comment-16466454 ] Hudson commented on HBASE-20523: Results for branch branch-1.4 [build #313 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/313/]: (x) *{color:red}-1 overall{color}* details (if available): (x) {color:red}-1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/313//General_Nightly_Build_Report/] (x) {color:red}-1 jdk7 checks{color} -- For more information [see jdk7 report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/313//JDK7_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/313//JDK8_Nightly_Build_Report_(Hadoop2)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 1.5.0, 1.2.7, 2.0.1, 1.4.5 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch, > HBASE-20523_branch-1.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466289#comment-16466289 ] stack commented on HBASE-20538: --- Thanks for reminder [~mdrob] > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: > 0001-HBASE-20538-TestSaslFanOutOneBlockAsyncDFSOutput-DISABLE.patch > > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20537) The download link is not available in the downloads webpage
[ https://issues.apache.org/jira/browse/HBASE-20537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-20537. --- Resolution: Fixed Assignee: stack Pushed small change adding 'src' and or 'bin' text. Thanks [~iemejia]. > The download link is not available in the downloads webpage > --- > > Key: HBASE-20537 > URL: https://issues.apache.org/jira/browse/HBASE-20537 > Project: HBase > Issue Type: Bug > Components: website >Affects Versions: 2.0.0 >Reporter: Ismaël Mejía >Assignee: stack >Priority: Major > > When you open https://hbase.apache.org/downloads.html > There is not a direct link to download both the binary or source > distributions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20257) hbase-spark should not depend on com.google.code.findbugs.jsr305
[ https://issues.apache.org/jira/browse/HBASE-20257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466182#comment-16466182 ] Ted Yu commented on HBASE-20257: Ping [~busbey] > hbase-spark should not depend on com.google.code.findbugs.jsr305 > > > Key: HBASE-20257 > URL: https://issues.apache.org/jira/browse/HBASE-20257 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 3.0.0 >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Minor > Labels: beginner > Attachments: HBASE-20257.v01.patch, HBASE-20257.v02.patch, > HBASE-20257.v03.patch, HBASE-20257.v04.patch > > > The following can be observed in the build output of master branch: > {code} > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed > with message: > We don't allow the JSR305 jar from the Findbugs project, see HBASE-16321. > Found Banned Dependency: com.google.code.findbugs:jsr305:jar:1.3.9 > Use 'mvn dependency:tree' to locate the source of the banned dependencies. > {code} > Here is related snippet from hbase-spark/pom.xml: > {code} > > com.google.code.findbugs > jsr305 > {code} > Dependency on jsr305 should be dropped. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20537) The download link is not available in the downloads webpage
[ https://issues.apache.org/jira/browse/HBASE-20537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466180#comment-16466180 ] stack commented on HBASE-20537: --- Funny, the little boxes will work... but yeah, should have more than this. Let me fix. > The download link is not available in the downloads webpage > --- > > Key: HBASE-20537 > URL: https://issues.apache.org/jira/browse/HBASE-20537 > Project: HBase > Issue Type: Bug > Components: website >Affects Versions: 2.0.0 >Reporter: Ismaël Mejía >Priority: Major > > When you open https://hbase.apache.org/downloads.html > There is not a direct link to download both the binary or source > distributions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466178#comment-16466178 ] stack commented on HBASE-20538: --- Disabled the test for now till dig in more (Need to fix HDFS-keymaking side?). Pushed to branch-2.0+. Doesn't seem to be issue in branch-1. > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: > 0001-HBASE-20538-TestSaslFanOutOneBlockAsyncDFSOutput-DISABLE.patch > > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20538: -- Fix Version/s: 2.0.1 2.1.0 3.0.0 > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: > 0001-HBASE-20538-TestSaslFanOutOneBlockAsyncDFSOutput-DISABLE.patch > > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20538: -- Attachment: 0001-HBASE-20538-TestSaslFanOutOneBlockAsyncDFSOutput-DISABLE.patch > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > Attachments: > 0001-HBASE-20538-TestSaslFanOutOneBlockAsyncDFSOutput-DISABLE.patch > > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466173#comment-16466173 ] Mike Drob commented on HBASE-20538: --- Our peter mentioned this during the 2.0 release vote as well > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20538: -- Priority: Critical (was: Major) > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466161#comment-16466161 ] stack commented on HBASE-20538: --- Here is exception: {code} java.lang.reflect.InvocationTargetException at org.apache.hadoop.hbase.io.asyncfs.TestSaslFanOutOneBlockAsyncDFSOutput.createEncryptionZone(TestSaslFanOutOneBlockAsyncDFSOutput.java:201) at org.apache.hadoop.hbase.io.asyncfs.TestSaslFanOutOneBlockAsyncDFSOutput.setUp(TestSaslFanOutOneBlockAsyncDFSOutput.java:229) Caused by: org.apache.hadoop.ipc.RemoteException: Can't recover key for test_key from keystore file:/testptch/hbase/hbase-server/target/test-data/75e90585-3745-4205-a94b-405265c6c96f/test.jks at org.apache.hadoop.crypto.key.JavaKeyStoreProvider.getMetadata(JavaKeyStoreProvider.java:424) at org.apache.hadoop.crypto.key.KeyProviderExtension.getMetadata(KeyProviderExtension.java:100) at org.apache.hadoop.hdfs.server.namenode.FSDirEncryptionZoneOp.ensureKeyIsInitialized(FSDirEncryptionZoneOp.java:124) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.createEncryptionZone(FSNamesystem.java:7002) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.createEncryptionZone(NameNodeRpcServer.java:2036) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.createEncryptionZone(ClientNamenodeProtocolServerSideTranslatorPB.java:1448) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675) Caused by: java.security.UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property at com.sun.crypto.provider.KeyProtector.unseal(KeyProtector.java:352) at com.sun.crypto.provider.JceKeyStore.engineGetKey(JceKeyStore.java:136) at java.security.KeyStore.getKey(KeyStore.java:1023) at org.apache.hadoop.crypto.key.JavaKeyStoreProvider.getMetadata(JavaKeyStoreProvider.java:410) ... 14 more at org.apache.hadoop.hbase.io.asyncfs.TestSaslFanOutOneBlockAsyncDFSOutput.createEncryptionZone(TestSaslFanOutOneBlockAsyncDFSOutput.java:201) at org.apache.hadoop.hbase.io.asyncfs.TestSaslFanOutOneBlockAsyncDFSOutput.setUp(TestSaslFanOutOneBlockAsyncDFSOutput.java:229) {code} > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
stack created HBASE-20538: - Summary: TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property Key: HBASE-20538 URL: https://issues.apache.org/jira/browse/HBASE-20538 Project: HBase Issue Type: Bug Reporter: stack Infra must have updated our JDK over the weekend. The test TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. [~gabor.bota] ran into it already over in HDFS-13494 and provided nice pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20537) The download link is not available in the downloads webpage
[ https://issues.apache.org/jira/browse/HBASE-20537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466097#comment-16466097 ] stack commented on HBASE-20537: --- duh > The download link is not available in the downloads webpage > --- > > Key: HBASE-20537 > URL: https://issues.apache.org/jira/browse/HBASE-20537 > Project: HBase > Issue Type: Bug > Components: website >Affects Versions: 2.0.0 >Reporter: Ismaël Mejía >Priority: Major > > When you open https://hbase.apache.org/downloads.html > There is not a direct link to download both the binary or source > distributions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466057#comment-16466057 ] Hudson commented on HBASE-20523: Results for branch branch-2.0 [build #270 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/270/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/270//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/270//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/270//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 1.5.0, 1.2.7, 2.0.1, 1.4.5 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch, > HBASE-20523_branch-1.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466038#comment-16466038 ] Hudson commented on HBASE-20523: Results for branch branch-2 [build #706 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/706/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/706//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/706//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/706//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 1.5.0, 1.2.7, 2.0.1, 1.4.5 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch, > HBASE-20523_branch-1.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20500) [rsgroup] should keep at least one server in default group
[ https://issues.apache.org/jira/browse/HBASE-20500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-20500: --- Status: Open (was: Patch Available) > [rsgroup] should keep at least one server in default group > -- > > Key: HBASE-20500 > URL: https://issues.apache.org/jira/browse/HBASE-20500 > Project: HBase > Issue Type: Bug > Components: rsgroup >Affects Versions: 2.0.0-beta-2 >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Major > Attachments: HBASE-20500-branch-2.v1.patch, > HBASE-20500-master.v1.patch, HBASE-20500-master.v2.patch, > HBASE-20500-master.v3.patch, HBASE-20500-master.v4.patch > > > we move all the servers from default group > the default group will has no servers, > then we create a new table , > it will failed case of the default group has no servers > eorr info is : > EROOR: ConstraintException:Target RSGroup must have at lease on server > > we should keep at least one server in 'default' RSGroup > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465881#comment-16465881 ] Ashish Singhi commented on HBASE-20004: --- Attached a patch making the option of enabling and disabling the OPTIONS method as configurable. By default OPTIONS method will not be added in constraint mapping and will be added in the other places. Please review. > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Attachments: HBASE-20004.branch-1.patch, HBASE-20004.patch, > HBASE-20004.v1.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-20004: -- Attachment: HBASE-20004.v1.patch > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Attachments: HBASE-20004.branch-1.patch, HBASE-20004.patch, > HBASE-20004.v1.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20259) Doc configs for in-memory-compaction and add detail to in-memory-compaction logging
[ https://issues.apache.org/jira/browse/HBASE-20259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465874#comment-16465874 ] Chia-Ping Tsai commented on HBASE-20259: The release notes says "Disables in-memory compaction as default." But the default policy still is BASIC. {code:java} public static final String COMPACTING_MEMSTORE_TYPE_DEFAULT = String.valueOf(MemoryCompactionPolicy.BASIC){code} Could someone point out the "Disables in-memory compaction as default." for me? Or setting BASIC be default is a kind of disabling the in-memory compaction since basic policy only invokes the flattening and merging? It seems to me HBASE-20464 does disable the in-memory compaction. > Doc configs for in-memory-compaction and add detail to in-memory-compaction > logging > --- > > Key: HBASE-20259 > URL: https://issues.apache.org/jira/browse/HBASE-20259 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-20259.branch-2.0.ADDENDUM.patch, > HBASE-20259.master.001.patch, HBASE-20259.master.002.patch, > HBASE-20259.master.003.patch > > > I set {{hbase.systemtables.compacting.memstore.type}} to NONE but it seems > like in-memory is still on. My table looks like this: > {code} > Table ycsb is ENABLED > ycsb > COLUMN FAMILIES DESCRIPTION > {NAME => 'family', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false', > NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE', > CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => > 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER = > > 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMORY => 'false', > > CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', > > COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} > {code} > Looks like table doesn't have it on either (IN_MEMORY_COMPACTION doesn't show > in the above). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
[ https://issues.apache.org/jira/browse/HBASE-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465857#comment-16465857 ] Hadoop QA commented on HBASE-20536: --- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 17s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 48s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 12s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m 1s{color} | {color:green} branch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 5s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 45s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 45s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 12s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m 3s{color} | {color:green} patch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 15m 5s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 21s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green}110m 58s{color} | {color:green} hbase-server in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 21s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}158m 43s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f | | JIRA Issue | HBASE-20536 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12922248/HBASE-20536.master.001.patch | | Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 05e7b80b7e99 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / 8e6ff689e8 | | maven | version: Apache Maven 3.5.3 (3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) | | Default Java | 1.8.0_171 | | findbugs | v3.1.0-RC3 | | Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/12737/testReport/ | | Max. process+thread count | 4305 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/12737/console | | Powered by | Apache Yetus 0.7.0 http://yetus.apache.org | This message was automatically generated. > Make Test
[jira] [Commented] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465849#comment-16465849 ] Hudson commented on HBASE-20523: Results for branch master [build #323 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/323/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/master/323//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/master/323//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/master/323//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 1.5.0, 1.2.7, 2.0.1, 1.4.5 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch, > HBASE-20523_branch-1.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20508) TestIncrementalBackupWithBulkLoad doesn't need to be Parameterized test
[ https://issues.apache.org/jira/browse/HBASE-20508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465848#comment-16465848 ] Hudson commented on HBASE-20508: Results for branch master [build #323 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/323/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/master/323//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/master/323//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/master/323//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > TestIncrementalBackupWithBulkLoad doesn't need to be Parameterized test > --- > > Key: HBASE-20508 > URL: https://issues.apache.org/jira/browse/HBASE-20508 > Project: HBase > Issue Type: Test > Components: backup&restore >Reporter: Ted Yu >Assignee: maoling >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-20508-master.v0.patch, HBASE-20508-master.v1.patch > > > TestIncrementalBackupWithBulkLoad currently is Parameterized with only one > value returned from data() method. > In its ctor, this value is ignored. > TestIncrementalBackupWithBulkLoad doesn't need to be Parameterized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20537) The download link is not available in the downloads webpage
Ismaël Mejía created HBASE-20537: Summary: The download link is not available in the downloads webpage Key: HBASE-20537 URL: https://issues.apache.org/jira/browse/HBASE-20537 Project: HBase Issue Type: Bug Components: website Affects Versions: 2.0.0 Reporter: Ismaël Mejía When you open https://hbase.apache.org/downloads.html There is not a direct link to download both the binary or source distributions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20475) Fix the flaky TestReplicationDroppedTables unit test.
[ https://issues.apache.org/jira/browse/HBASE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465790#comment-16465790 ] Zheng Hu commented on HBASE-20475: -- I guess we were stopping a ReplicationSourceShipper which has not finished its initialization. so the NPE happen {code} worker.entryReader.interrupt(); {code} > Fix the flaky TestReplicationDroppedTables unit test. > - > > Key: HBASE-20475 > URL: https://issues.apache.org/jira/browse/HBASE-20475 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20475-addendum-v2.patch, > HBASE-20475-addendum-v3.patch, HBASE-20475-addendum.patch, HBASE-20475.patch > > > See > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20475) Fix the flaky TestReplicationDroppedTables unit test.
[ https://issues.apache.org/jira/browse/HBASE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465759#comment-16465759 ] Zheng Hu edited comment on HBASE-20475 at 5/7/18 10:46 AM: --- Checked the UT & log again, the phenomenon is: {code:java} > Key: HBASE-20475 > URL: https://issues.apache.org/jira/browse/HBASE-20475 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20475-addendum-v2.patch, > HBASE-20475-addendum-v3.patch, HBASE-20475-addendum.patch, HBASE-20475.patch > > > See > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20475) Fix the flaky TestReplicationDroppedTables unit test.
[ https://issues.apache.org/jira/browse/HBASE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465759#comment-16465759 ] Zheng Hu edited comment on HBASE-20475 at 5/7/18 10:43 AM: --- Checked the UT & log again, the phenomenon is: {code:java} > Key: HBASE-20475 > URL: https://issues.apache.org/jira/browse/HBASE-20475 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20475-addendum-v2.patch, > HBASE-20475-addendum-v3.patch, HBASE-20475-addendum.patch, HBASE-20475.patch > > > See > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20475) Fix the flaky TestReplicationDroppedTables unit test.
[ https://issues.apache.org/jira/browse/HBASE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465766#comment-16465766 ] Duo Zhang commented on HBASE-20475: --- What about the NPE posted by me above? Is there a race? > Fix the flaky TestReplicationDroppedTables unit test. > - > > Key: HBASE-20475 > URL: https://issues.apache.org/jira/browse/HBASE-20475 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20475-addendum-v2.patch, > HBASE-20475-addendum-v3.patch, HBASE-20475-addendum.patch, HBASE-20475.patch > > > See > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
[ https://issues.apache.org/jira/browse/HBASE-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465760#comment-16465760 ] Duo Zhang commented on HBASE-20536: --- +1. > Make TestRegionServerAccounting stable and it should not use absolute number > > > Key: HBASE-20536 > URL: https://issues.apache.org/jira/browse/HBASE-20536 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-20536.master.001.patch > > > TestRegionServerAccounting failed on our internal jenkin job as we config Xmx > to 10G. We should modify the absolute number to relative value. > {code:java} > new MemStoreSize((3L * 1024L * 1024L * 1024L), (1L * 1024L * 1024L * 1024L), > 0);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20475) Fix the flaky TestReplicationDroppedTables unit test.
[ https://issues.apache.org/jira/browse/HBASE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465759#comment-16465759 ] Zheng Hu commented on HBASE-20475: -- Checked the UT & log again, the phenomenon is: {code} > Key: HBASE-20475 > URL: https://issues.apache.org/jira/browse/HBASE-20475 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20475-addendum-v2.patch, > HBASE-20475-addendum-v3.patch, HBASE-20475-addendum.patch, HBASE-20475.patch > > > See > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
[ https://issues.apache.org/jira/browse/HBASE-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-20536: --- Assignee: Guanghao Zhang Status: Patch Available (was: Open) > Make TestRegionServerAccounting stable and it should not use absolute number > > > Key: HBASE-20536 > URL: https://issues.apache.org/jira/browse/HBASE-20536 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-20536.master.001.patch > > > TestRegionServerAccounting failed on our internal jenkin job as we config Xmx > to 10G. We should modify the absolute number to relative value. > {code:java} > new MemStoreSize((3L * 1024L * 1024L * 1024L), (1L * 1024L * 1024L * 1024L), > 0);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
[ https://issues.apache.org/jira/browse/HBASE-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-20536: --- Attachment: HBASE-20536.master.001.patch > Make TestRegionServerAccounting stable and it should not use absolute number > > > Key: HBASE-20536 > URL: https://issues.apache.org/jira/browse/HBASE-20536 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Priority: Minor > Attachments: HBASE-20536.master.001.patch > > > TestRegionServerAccounting failed on our internal jenkin job as we config Xmx > to 10G. We should modify the absolute number to relative value. > {code:java} > new MemStoreSize((3L * 1024L * 1024L * 1024L), (1L * 1024L * 1024L * 1024L), > 0);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465676#comment-16465676 ] Hudson commented on HBASE-20523: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1109 (See [https://builds.apache.org/job/HBase-1.2-IT/1109/]) HBASE-20523 PE tool should support configuring client side buffering (ramkrishna.s.vasudevan: rev b3a162bc11d5eb76479b45b07efc5cbd3bc6de47) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 1.5.0, 1.2.7, 2.0.1, 1.4.5 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch, > HBASE-20523_branch-1.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-20523: --- Resolution: Fixed Fix Version/s: 1.4.5 1.2.7 1.5.0 Status: Resolved (was: Patch Available) Pushed to master, 2.0, branch-2, 1.2, 1.4 and branch-1. Thanks all for the reviews. > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 1.5.0, 1.2.7, 2.0.1, 1.4.5 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch, > HBASE-20523_branch-1.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465652#comment-16465652 ] Hadoop QA commented on HBASE-20523: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s{color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 3s{color} | {color:red} HBASE-20523 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | HBASE-20523 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12922244/HBASE-20523_branch-1.patch | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/12736/console | | Powered by | Apache Yetus 0.7.0 http://yetus.apache.org | This message was automatically generated. > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 2.0.1 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch, > HBASE-20523_branch-1.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-20523: --- Attachment: HBASE-20523_branch-1.patch > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 2.0.1 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch, > HBASE-20523_branch-1.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-20523: --- Attachment: HBASE-20523_branch-1.2.patch > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 2.0.1 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch, > HBASE-20523_branch-1.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-20523: --- Attachment: HBASE-20523_branch-1.4.patch > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 2.0.1 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.4.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20536) Make TestRegionServerAccounting stable and it should not use absolute number
Guanghao Zhang created HBASE-20536: -- Summary: Make TestRegionServerAccounting stable and it should not use absolute number Key: HBASE-20536 URL: https://issues.apache.org/jira/browse/HBASE-20536 Project: HBase Issue Type: Improvement Reporter: Guanghao Zhang TestRegionServerAccounting failed on our internal jenkin job as we config Xmx to 10G. We should modify the absolute number to relative value. {code:java} new MemStoreSize((3L * 1024L * 1024L * 1024L), (1L * 1024L * 1024L * 1024L), 0);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18812) Recategorize some of classes used as tools
[ https://issues.apache.org/jira/browse/HBASE-18812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465629#comment-16465629 ] Guangxu Cheng commented on HBASE-18812: --- # BackupDriver # RestoreDriver # MapReduceHFileSplitterJob # Export # IndexBuilder # SampleUploader # StripeCompactionsPerformanceEvaluation # HashTable # SyncTable # VerifyReplication # ProcedureWALLoaderPerformanceEvaluation # ProcedureWALPerformanceEvaluation # ExpiredMobFileCleaner # Compressor # DumpReplicationQueues # ReplicationSyncUp # CreateSnapshot # Canary # ServerCommandLine # MasterProcedureSchedulerPerformanceEvaluation # WALPerformanceEvaluation # ZKAclReset Attach 002 patch as [~chia7712] suggestions.Changing the InterfaceAudience from Private to LP. Thanks > Recategorize some of classes used as tools > -- > > Key: HBASE-18812 > URL: https://issues.apache.org/jira/browse/HBASE-18812 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Guangxu Cheng >Priority: Major > Attachments: HBASE-18812.master.001.patch, > HBASE-18812.master.002.patch > > > The classes used from cmd line should be made as LimitedPrivate.TOOLS. The > candidates are shown below. > # BackupDriver > # RestoreDriver > # CreateSnapshot > # SnapshotInfo > # ExportSnapshot > # Canary > # VersionInfo > # RegionMover > # CellCounter > # CopyTable > # DumpReplicationQueues > # Export > # HashTable > # Import > # ImportTsv > # LoadIncrementalHFiles > # ReplicationSyncUp > # SyncTable > # VerifyReplication > # WALPlayer > # ZkAclReset -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18812) Recategorize some of classes used as tools
[ https://issues.apache.org/jira/browse/HBASE-18812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guangxu Cheng updated HBASE-18812: -- Attachment: HBASE-18812.master.002.patch > Recategorize some of classes used as tools > -- > > Key: HBASE-18812 > URL: https://issues.apache.org/jira/browse/HBASE-18812 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Guangxu Cheng >Priority: Major > Attachments: HBASE-18812.master.001.patch, > HBASE-18812.master.002.patch > > > The classes used from cmd line should be made as LimitedPrivate.TOOLS. The > candidates are shown below. > # BackupDriver > # RestoreDriver > # CreateSnapshot > # SnapshotInfo > # ExportSnapshot > # Canary > # VersionInfo > # RegionMover > # CellCounter > # CopyTable > # DumpReplicationQueues > # Export > # HashTable > # Import > # ImportTsv > # LoadIncrementalHFiles > # ReplicationSyncUp > # SyncTable > # VerifyReplication > # WALPlayer > # ZkAclReset -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20521) TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script run fail
[ https://issues.apache.org/jira/browse/HBASE-20521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465586#comment-16465586 ] Michael Jin commented on HBASE-20521: - [~mdrob], any update about the attach? > TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script > run fail > --- > > Key: HBASE-20521 > URL: https://issues.apache.org/jira/browse/HBASE-20521 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.0.0 > Environment: spark 2.2.1, hbase 2.0.0 >Reporter: Michael Jin >Assignee: Michael Jin >Priority: Major > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: HBASE-20521.master.001.patch, > HBASE-20521.master.002.patch > > > HBASE-20295 fix null point exception of "conf" member variable, add > "context.getConfiguration()" in case when "conf" object was not been properly > initialized, and put it into the first priority checking sequence, this code > change affect user call "setConf" explicitly initialize "conf" object in > TableOutputFormat object, proposal to change checking sequence, use "conf" > object from "getConf" method first . -- This message was sent by Atlassian JIRA (v7.6.3#76005)