[GitHub] [hbase] Apache-HBase commented on issue #645: HBASE-22930 Set unique name to longCompactions/shortCompactions threads
Apache-HBase commented on issue #645: HBASE-22930 Set unique name to longCompactions/shortCompactions threads URL: https://github.com/apache/hbase/pull/645#issuecomment-533418496 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 1m 56s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :yellow_heart: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | :green_heart: | mvninstall | 8m 26s | branch-1 passed | | :green_heart: | compile | 2m 20s | branch-1 passed with JDK v1.8.0_222 | | :broken_heart: | compile | 0m 22s | hbase-server in branch-1 failed with JDK v1.7.0_232. | | :green_heart: | checkstyle | 1m 45s | branch-1 passed | | :green_heart: | shadedjars | 3m 11s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 37s | branch-1 passed with JDK v1.8.0_222 | | :green_heart: | javadoc | 0m 43s | branch-1 passed with JDK v1.7.0_232 | | :blue_heart: | spotbugs | 3m 6s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 3m 2s | branch-1 passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 2m 10s | the patch passed | | :green_heart: | compile | 2m 5s | the patch passed with JDK v1.8.0_222 | | :green_heart: | javac | 2m 5s | the patch passed | | :broken_heart: | compile | 0m 24s | hbase-server in the patch failed with JDK v1.7.0_232. | | :broken_heart: | javac | 0m 24s | hbase-server in the patch failed with JDK v1.7.0_232. | | :green_heart: | checkstyle | 1m 43s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 3m 19s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 5m 44s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | :green_heart: | javadoc | 0m 32s | the patch passed with JDK v1.8.0_222 | | :green_heart: | javadoc | 0m 49s | the patch passed with JDK v1.7.0_232 | | :green_heart: | findbugs | 3m 35s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 151m 41s | hbase-server in the patch passed. | | :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. | | | | 198m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-645/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/645 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 7cc6dc835d2f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-645/out/precommit/personality/provided.sh | | git revision | branch-1 / 5c4d8e0 | | Default Java | 1.7.0_232 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_222 /usr/lib/jvm/zulu-7-amd64:1.7.0_232 | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-645/1/artifact/out/branch-compile-hbase-server-jdk1.7.0_232.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-645/1/artifact/out/patch-compile-hbase-server-jdk1.7.0_232.txt | | javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-645/1/artifact/out/patch-compile-hbase-server-jdk1.7.0_232.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-645/1/testReport/ | | Max. process+thread count | 5035 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-645/1/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git
[GitHub] [hbase] saintstack opened a new pull request #646: HBASE-23055 Alter hbase:meta
saintstack opened a new pull request #646: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/646 Make it so hbase:meta can be altered. TableState for hbase:meta is kept in Master. State is in-memory transient so if Master fails, hbase:meta is ENABLED again. hbase:meta schema will be bootstrapped from the filesystem. Changes to filesystem schema are atomic so we should be ok if Master fails mid-edit (TBD) Undoes a bunch of guards that prevented our being able to edit hbase:meta. At minimmum, need to add in a bunch of WARNING. TODO: Tests, more clarity around hbase:meta table state, and undoing references to hard-coded hbase:meta regioninfo. M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java Throw illegal access exception if you try to use MetaTableAccessor getting state of the hbase:meta table. M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java TODO: getTableState needs work in Connection implemetnations. Presumes state is in meta table for all tables. Uses MetaTableAccessor. TODO: More cleanup in here and more cleanup in async versions. M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java Change isTableDisabled/Enabled implementation to ask the Master instead. This will give the Master's TableStateManager's opinion rather than client figuring it for themselves reading meta table direct. M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java TODO: Cleanup in here. Go to master for state, not to meta. M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java Logging cleanup. M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java Shutdown access. M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java Just cleanup. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java Add state holder for hbase:meta. Removed unused methods. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java Shut down access. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java Allow hbase:meta to be disabled. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java Allow hbase:meta to be enabled. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #644: HBASE-23055 Alter hbase:meta
saintstack commented on issue #644: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/644#issuecomment-533417470 I see. It went against master branch instead of branch-2. Let me fix. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack closed pull request #644: HBASE-23055 Alter hbase:meta
saintstack closed pull request #644: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/644 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #644: HBASE-23055 Alter hbase:meta
saintstack commented on issue #644: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/644#issuecomment-533417126 Um. Mangled. Closing. Will open new one. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22941) MetaTableAccessor.getMergeRegions() returns parent regions in random order
[ https://issues.apache.org/jira/browse/HBASE-22941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934071#comment-16934071 ] stack commented on HBASE-22941: --- Bad backport or extra assert in that branch? Thanks [~zhangduo] > MetaTableAccessor.getMergeRegions() returns parent regions in random order > -- > > Key: HBASE-22941 > URL: https://issues.apache.org/jira/browse/HBASE-22941 > Project: HBase > Issue Type: Bug >Affects Versions: master >Reporter: István Tóth >Assignee: István Tóth >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.1, 2.1.7 > > > The multi-region merge patch has changed the behaviour of the > MetaTableAccessor.getMergeRegions() method. Earlier, the parent regions were > returned in the same order that they were passed to mergeRegion(). > The patch has changed this to return the regions in random order. To be more > precise, the parent regions are stored in random order in Meta. > I propose storing and returning the parent regions in ascending order. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23015) branch-1 hbase-server, testing util, and shaded testing util need jackson
[
https://issues.apache.org/jira/browse/HBASE-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934038#comment-16934038
]
Sean Busbey commented on HBASE-23015:
-
FWIW, I have been focusing on getting 1.3/1.4 out prior to getting to 1.5
because they have a critical bug that needs to get into a release. If you're up
for RM'ing a 1.5 release in parallel once this blocker closes that'd be
wonderful.
> branch-1 hbase-server, testing util, and shaded testing util need jackson
> --
>
> Key: HBASE-23015
> URL: https://issues.apache.org/jira/browse/HBASE-23015
> Project: HBase
> Issue Type: Bug
> Components: Client, shading
>Affects Versions: 1.5.0, 1.3.6, 1.4.11
>Reporter: Sean Busbey
>Assignee: Viraj Jasani
>Priority: Blocker
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-23015.branch-1.3.000.patch
>
>
> HBASE-22728 moved out jackson transitive dependencies. mostly good, but
> moving jackson2 to provided in hbase-server broke few things
> testing-util needs a transitive jackson 2 in order to start the minicluster,
> currently fails with CNFE for {{com.fasterxml.jackson.databind.ObjectMapper}}
> when trying to initialize the master.
> shaded-testing-util needs a relocated jackson 2 for the same reason
> it's not used for any of the mapreduce stuff in hbase-server, so
> {{hbase-shaded-server}} for that purpose should be fine. But it is used by
> {{WALPrettyPrinter}} and some folks might expect that to work from that
> artifact since it is present.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23052) hbase-thirdparty version of GSON that works for branch-1
[ https://issues.apache.org/jira/browse/HBASE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934036#comment-16934036 ] Sean Busbey commented on HBASE-23052: - Third option: make a [jdk7 compat modules that has misc dependencies, starting with gson PR#5|https://github.com/apache/hbase-thirdparty/pull/5] I like this one too. Again we don't downgrade anything. We have a straightforward path for adding to the things relocated for branches-1 and handling the case if we need to diverge on the versions around for jdk7 and jdk8. it's unfortunate that we end up publishing multiple copies of the same dependency before we actually need to though. > hbase-thirdparty version of GSON that works for branch-1 > > > Key: HBASE-23052 > URL: https://issues.apache.org/jira/browse/HBASE-23052 > Project: HBase > Issue Type: Improvement > Components: dependencies >Affects Versions: thirdparty-2.2.1 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > > HBASE-23015 is buttoning up a needed move off of jackson 1 in branches-1. > We've already got the implementation work in place to move onto the > hbase-thirdparty relocated GSON, but we can't currently build because other > dependencies included in the miscellaneous module is JDK8+ only and branch-1 > needs to work for jdk7. > couple of options: > * make the entire hbase-thirdparty repo work with jdk7 > * break out gson from the clearing house miscellaneous module and make *just* > the new gson module jdk7 compatible > * make a jdk7 compatible miscellaneous module and move gson over there (in > case we decide to move branch-1 off of other problematic libraries e.g. guava) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ZhaoBQ commented on issue #633: HBASE-22890 Verify the file integrity in persistent IOEngine
ZhaoBQ commented on issue #633: HBASE-22890 Verify the file integrity in persistent IOEngine URL: https://github.com/apache/hbase/pull/633#issuecomment-533399535 @Reidd @anoopsjohn Thank you for your patience and advice！ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Reidddddd commented on a change in pull request #615: HBASE-22975 Add read and write QPS metrics at server level and table level
Reidd commented on a change in pull request #615: HBASE-22975 Add read and write QPS metrics at server level and table level URL: https://github.com/apache/hbase/pull/615#discussion_r326466489 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java ## @@ -6379,7 +6382,6 @@ String prepareBulkLoad(byte[] family, String srcPath, boolean copyFile) } } } - closeBulkRegionOperation(); } return isSuccessful ? storeFiles : null; Review comment: Ditto This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Reidddddd commented on a change in pull request #615: HBASE-22975 Add read and write QPS metrics at server level and table level
Reidd commented on a change in pull request #615: HBASE-22975 Add read and
write QPS metrics at server level and table level
URL: https://github.com/apache/hbase/pull/615#discussion_r326466480
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -6201,7 +6205,6 @@ String prepareBulkLoad(byte[] family, String srcPath,
boolean copyFile)
boolean isSuccessful = false;
try {
this.writeRequestsCount.increment();
-
Review comment:
Please avoid unnecessary change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22941) MetaTableAccessor.getMergeRegions() returns parent regions in random order
[ https://issues.apache.org/jira/browse/HBASE-22941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934030#comment-16934030 ] Duo Zhang commented on HBASE-22941: --- OK found another problem when backporting to branch-2.1, seems we have HBASE-22777 in place, but in MasterRpcServices.mergeTableRegions we still have a assert which assume that there are only two regions to merge... > MetaTableAccessor.getMergeRegions() returns parent regions in random order > -- > > Key: HBASE-22941 > URL: https://issues.apache.org/jira/browse/HBASE-22941 > Project: HBase > Issue Type: Bug >Affects Versions: master >Reporter: István Tóth >Assignee: István Tóth >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.1, 2.1.7 > > > The multi-region merge patch has changed the behaviour of the > MetaTableAccessor.getMergeRegions() method. Earlier, the parent regions were > returned in the same order that they were passed to mergeRegion(). > The patch has changed this to return the regions in random order. To be more > precise, the parent regions are stored in random order in Meta. > I propose storing and returning the parent regions in ascending order. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] anoopsjohn commented on a change in pull request #645: HBASE-22930 Set unique name to longCompactions/shortCompactions threads
anoopsjohn commented on a change in pull request #645: HBASE-22930 Set unique
name to longCompactions/shortCompactions threads
URL: https://github.com/apache/hbase/pull/645#discussion_r326464585
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
##
@@ -131,11 +132,12 @@ public Thread newThread(Runnable r) {
private void createSplitExcecutors() {
final String n = Thread.currentThread().getName();
int splitThreads = conf.getInt(SPLIT_THREADS, SPLIT_THREADS_DEFAULT);
+final AtomicInteger splitThreadCounter = new AtomicInteger(0);
this.splits =
(ThreadPoolExecutor) Executors.newFixedThreadPool(splitThreads, new
ThreadFactory() {
@Override
Review comment:
And same way below also.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] anoopsjohn commented on a change in pull request #645: HBASE-22930 Set unique name to longCompactions/shortCompactions threads
anoopsjohn commented on a change in pull request #645: HBASE-22930 Set unique
name to longCompactions/shortCompactions threads
URL: https://github.com/apache/hbase/pull/645#discussion_r326464413
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
##
@@ -131,11 +132,12 @@ public Thread newThread(Runnable r) {
private void createSplitExcecutors() {
final String n = Thread.currentThread().getName();
int splitThreads = conf.getInt(SPLIT_THREADS, SPLIT_THREADS_DEFAULT);
+final AtomicInteger splitThreadCounter = new AtomicInteger(0);
this.splits =
(ThreadPoolExecutor) Executors.newFixedThreadPool(splitThreads, new
ThreadFactory() {
@Override
Review comment:
+1 on what Reid said.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (HBASE-23051) Remove unneeded Mockito.mock invocations
[
https://issues.apache.org/jira/browse/HBASE-23051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-23051:
--
Attachment: HBASE-23051-branch-2-addendum.patch
> Remove unneeded Mockito.mock invocations
>
>
> Key: HBASE-23051
> URL: https://issues.apache.org/jira/browse/HBASE-23051
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
> Attachments: HBASE-23051-branch-2-addendum.patch
>
>
> ErrorProne fails the build using the new Mockito version.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile
> (default-testCompile) on project hbase-server: Compilation failure:
> Compilation failure:
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/MockMasterServices.java:[147,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedureControllers.java:[331,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR] -> [Help 1] {noformat}
> [https://builds.apache.org/job/HBase%20Nightly/job/master/1456/artifact/output-jdk8-hadoop2/patch-compile-root.txt]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-23051) Remove unneeded Mockito.mock invocations
[
https://issues.apache.org/jira/browse/HBASE-23051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-23051.
---
Hadoop Flags: Reviewed
Resolution: Fixed
> Remove unneeded Mockito.mock invocations
>
>
> Key: HBASE-23051
> URL: https://issues.apache.org/jira/browse/HBASE-23051
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
> Attachments: HBASE-23051-branch-2-addendum.patch
>
>
> ErrorProne fails the build using the new Mockito version.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile
> (default-testCompile) on project hbase-server: Compilation failure:
> Compilation failure:
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/MockMasterServices.java:[147,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedureControllers.java:[331,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR] -> [Help 1] {noformat}
> [https://builds.apache.org/job/HBase%20Nightly/job/master/1456/artifact/output-jdk8-hadoop2/patch-compile-root.txt]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HBASE-22873) Typo in block caching docs
[ https://issues.apache.org/jira/browse/HBASE-22873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-22873: - Assignee: Shuai Lin > Typo in block caching docs > -- > > Key: HBASE-22873 > URL: https://issues.apache.org/jira/browse/HBASE-22873 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Shuai Lin >Assignee: Shuai Lin >Priority: Minor > Fix For: 3.0.0 > > Attachments: hbase-22873.patch > > > To turn off block cache for a scan, setCacheBlocks should be used, but the > hbase book says "setCaching", which is not relevant. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23017) [Forward-port] Verify the file integrity in persistent IOEngine
[ https://issues.apache.org/jira/browse/HBASE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-23017: --- Issue Type: Improvement (was: Bug) > [Forward-port] Verify the file integrity in persistent IOEngine > --- > > Key: HBASE-23017 > URL: https://issues.apache.org/jira/browse/HBASE-23017 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Affects Versions: 2.2.0, master >Reporter: zbq.dean >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Verify the persistent cache file integrity before retrieve from persistence > file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23017) [Forward-port] Verify the file integrity in persistent IOEngine
[ https://issues.apache.org/jira/browse/HBASE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-23017: --- Fix Version/s: (was: master) 3.0.0 > [Forward-port] Verify the file integrity in persistent IOEngine > --- > > Key: HBASE-23017 > URL: https://issues.apache.org/jira/browse/HBASE-23017 > Project: HBase > Issue Type: Bug > Components: BucketCache >Affects Versions: 2.2.0, master >Reporter: zbq.dean >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Verify the persistent cache file integrity before retrieve from persistence > file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23017) [Forward-port] Verify the file integrity in persistent IOEngine
[ https://issues.apache.org/jira/browse/HBASE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-23017: --- Fix Version/s: master 2.3.0 > [Forward-port] Verify the file integrity in persistent IOEngine > --- > > Key: HBASE-23017 > URL: https://issues.apache.org/jira/browse/HBASE-23017 > Project: HBase > Issue Type: Bug > Components: BucketCache >Affects Versions: 2.2.0, master >Reporter: zbq.dean >Priority: Major > Fix For: 2.3.0, master > > > Verify the persistent cache file integrity before retrieve from persistence > file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23017) [Forward-port] Verify the file integrity in persistent IOEngine
[ https://issues.apache.org/jira/browse/HBASE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934027#comment-16934027 ] Anoop Sam John commented on HBASE-23017: HBASE-22890 is ready to go in now.. Pls change the master patch also accordingly. > [Forward-port] Verify the file integrity in persistent IOEngine > --- > > Key: HBASE-23017 > URL: https://issues.apache.org/jira/browse/HBASE-23017 > Project: HBase > Issue Type: Bug > Components: BucketCache >Affects Versions: 2.2.0, master >Reporter: zbq.dean >Priority: Major > > Verify the persistent cache file integrity before retrieve from persistence > file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-22890) Verify the files when RegionServer is starting and BucketCache is in file mode
[ https://issues.apache.org/jira/browse/HBASE-22890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934025#comment-16934025 ] Anoop Sam John commented on HBASE-22890: Please add ReleaseNotes also. > Verify the files when RegionServer is starting and BucketCache is in file mode > -- > > Key: HBASE-22890 > URL: https://issues.apache.org/jira/browse/HBASE-22890 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Affects Versions: 1.4.6, 1.4.10, 1.3.5 >Reporter: zbq.dean >Priority: Major > Fix For: 1.5.0, 1.4.11 > > Attachments: HBASE-22890.branch-1.0001.patch > > > We use FileIOEngine in BucketCache and have configured > . If the RegionServer is stopped, and the > cache data file or the backingMap persistence file is deleted or changed > before the RegionServer is restarted. When restart RegionServer, the > backingMap and cache data are Inconsistent, and get or scan command will get > IOException like follows: > bucket.BucketCache: Failed reading block > 18951416e2ca4e8bbc11523f4f5ea576_1125745279 from bucket cache > java.io.IOException: Invalid HFile block magic: > \x00\x00\x00\x00\x00\x00\x00\x00 > at org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:154) > at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:167) > at org.apache.hadoop.hbase.io.hfile.HFileBlock.(HFileBlock.java:347) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$1.deserialize(HFileBlock.java:247) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$1.deserialize(HFileBlock.java:226) > at > org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.getBlock(BucketCache.java:514) > .. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22890) Verify the files when RegionServer is starting and BucketCache is in file mode
[ https://issues.apache.org/jira/browse/HBASE-22890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-22890: --- Issue Type: Improvement (was: Bug) > Verify the files when RegionServer is starting and BucketCache is in file mode > -- > > Key: HBASE-22890 > URL: https://issues.apache.org/jira/browse/HBASE-22890 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Affects Versions: 1.4.6, 1.4.10, 1.3.5 >Reporter: zbq.dean >Priority: Major > Fix For: 1.5.0, 1.4.11 > > Attachments: HBASE-22890.branch-1.0001.patch > > > We use FileIOEngine in BucketCache and have configured > . If the RegionServer is stopped, and the > cache data file or the backingMap persistence file is deleted or changed > before the RegionServer is restarted. When restart RegionServer, the > backingMap and cache data are Inconsistent, and get or scan command will get > IOException like follows: > bucket.BucketCache: Failed reading block > 18951416e2ca4e8bbc11523f4f5ea576_1125745279 from bucket cache > java.io.IOException: Invalid HFile block magic: > \x00\x00\x00\x00\x00\x00\x00\x00 > at org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:154) > at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:167) > at org.apache.hadoop.hbase.io.hfile.HFileBlock.(HFileBlock.java:347) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$1.deserialize(HFileBlock.java:247) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$1.deserialize(HFileBlock.java:226) > at > org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.getBlock(BucketCache.java:514) > .. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] anoopsjohn commented on a change in pull request #633: HBASE-22890 Verify the file integrity in persistent IOEngine
anoopsjohn commented on a change in pull request #633: HBASE-22890 Verify the
file integrity in persistent IOEngine
URL: https://github.com/apache/hbase/pull/633#discussion_r326462840
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java
##
@@ -68,15 +97,18 @@ public FileIOEngine(long capacity, String... filePaths)
throws IOException {
// The next setting length will throw exception,logging this message
// is just used for the detail reason of exception，
String msg = "Only " + StringUtils.byteDesc(totalSpace)
- + " total space under " + filePath + ", not enough for requested
"
- + StringUtils.byteDesc(sizePerFile);
++ " total space under " + filePath + ", not enough for requested "
++ StringUtils.byteDesc(sizePerFile);
LOG.warn(msg);
}
-rafs[i].setLength(sizePerFile);
+File file = new File(filePath);
+if (file.length() != sizePerFile) {
+ rafs[i].setLength(sizePerFile);
+}
Review comment:
Yaya that was my Q.. Oh great.. I did not read whole the code.. Then its
perfect. Pls add enough comments around to know why this check and set length
is so imp. Great.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #616: HBASE-23015 : Moving from Jackson2 to shaded Gson (Backport HBASE-20587)
virajjasani commented on a change in pull request #616: HBASE-23015 : Moving from Jackson2 to shaded Gson (Backport HBASE-20587) URL: https://github.com/apache/hbase/pull/616#discussion_r326461597 ## File path: hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestOperation.java ## @@ -60,6 +59,9 @@ import org.apache.hadoop.hbase.testclassification.SmallTests; import org.apache.hadoop.hbase.util.BuilderStyleTest; import org.apache.hadoop.hbase.util.Bytes; +import org.apache.hadoop.hbase.util.GsonUtil; +import org.apache.hbase.thirdparty.com.google.common.reflect.TypeToken; Review comment: It has something to do with jdk7 compatibility or some issue in general? Sure, Let me take care of this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23052) hbase-thirdparty version of GSON that works for branch-1
[ https://issues.apache.org/jira/browse/HBASE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934017#comment-16934017 ] Sean Busbey commented on HBASE-23052: - Second option: make it so [hbase-thirdparty's gson is in its own jdk7-compat module PR#4|https://github.com/apache/hbase-thirdparty/pull/4] I like this one even though it's a major version bump on hbase-thirdparty. We don't have to downgrade anything and we aren't constrained on future dependency upgrades unless gson eventually puts out a jdk8+ only version (it's currently jdk6+) > hbase-thirdparty version of GSON that works for branch-1 > > > Key: HBASE-23052 > URL: https://issues.apache.org/jira/browse/HBASE-23052 > Project: HBase > Issue Type: Improvement > Components: dependencies >Affects Versions: thirdparty-2.2.1 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > > HBASE-23015 is buttoning up a needed move off of jackson 1 in branches-1. > We've already got the implementation work in place to move onto the > hbase-thirdparty relocated GSON, but we can't currently build because other > dependencies included in the miscellaneous module is JDK8+ only and branch-1 > needs to work for jdk7. > couple of options: > * make the entire hbase-thirdparty repo work with jdk7 > * break out gson from the clearing house miscellaneous module and make *just* > the new gson module jdk7 compatible > * make a jdk7 compatible miscellaneous module and move gson over there (in > case we decide to move branch-1 off of other problematic libraries e.g. guava) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] virajjasani edited a comment on issue #611: HBASE-22903 : Table to RegionStatesCount metrics
virajjasani edited a comment on issue #611: HBASE-22903 : Table to RegionStatesCount metrics URL: https://github.com/apache/hbase/pull/611#issuecomment-530481449 Please review @saintstack I tried to cover shell triggered and non-shell triggered alter schema update. If we have this metrics, probably good to use it for alter_schema command? Can also be used for other client get status This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #616: HBASE-23015 : Moving from Jackson2 to shaded Gson (Backport HBASE-20587)
busbey commented on a change in pull request #616: HBASE-23015 : Moving from Jackson2 to shaded Gson (Backport HBASE-20587) URL: https://github.com/apache/hbase/pull/616#discussion_r326460474 ## File path: hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestOperation.java ## @@ -60,6 +59,9 @@ import org.apache.hadoop.hbase.testclassification.SmallTests; import org.apache.hadoop.hbase.util.BuilderStyleTest; import org.apache.hadoop.hbase.util.Bytes; +import org.apache.hadoop.hbase.util.GsonUtil; +import org.apache.hbase.thirdparty.com.google.common.reflect.TypeToken; Review comment: while working on HBASE-23052 I realized this is the wrong `TypeToken`. should be `com.google.gson.reflect.TypeToken` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani edited a comment on issue #611: HBASE-22903 : Table to RegionStatesCount metrics
virajjasani edited a comment on issue #611: HBASE-22903 : Table to RegionStatesCount metrics URL: https://github.com/apache/hbase/pull/611#issuecomment-530481449 Please review @saintstack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Reopened] (HBASE-23051) Remove unneeded Mockito.mock invocations
[
https://issues.apache.org/jira/browse/HBASE-23051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang reopened HBASE-23051:
---
This breaks branch-2, missed a import.
> Remove unneeded Mockito.mock invocations
>
>
> Key: HBASE-23051
> URL: https://issues.apache.org/jira/browse/HBASE-23051
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> ErrorProne fails the build using the new Mockito version.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile
> (default-testCompile) on project hbase-server: Compilation failure:
> Compilation failure:
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/MockMasterServices.java:[147,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedureControllers.java:[331,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR] -> [Help 1] {noformat}
> [https://builds.apache.org/job/HBase%20Nightly/job/master/1456/artifact/output-jdk8-hadoop2/patch-compile-root.txt]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HBASE-23015) branch-1 hbase-server, testing util, and shaded testing util need jackson
[
https://issues.apache.org/jira/browse/HBASE-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934004#comment-16934004
]
Lars Hofhansl edited comment on HBASE-23015 at 9/20/19 3:22 AM:
I meant apply *locally* above. In any case it does not apply cleanly in my
setup, so it'd be a chunk of work anyway.
And totally agree that non-released branches are at downstream project's own
risk.
I will say that we have been dragging our feet on 1.5 trying to make it perfect
instead of just releasing (as long as there are *no new* problems) and fix
outstanding in the next release (i.e. 1.5.1) which should only be a month away.
was (Author: lhofhansl):
I meant apply *locally* above. In any case it does not apply cleanly in my
setup, so it'd be a chunk of work anyway.
> branch-1 hbase-server, testing util, and shaded testing util need jackson
> --
>
> Key: HBASE-23015
> URL: https://issues.apache.org/jira/browse/HBASE-23015
> Project: HBase
> Issue Type: Bug
> Components: Client, shading
>Affects Versions: 1.5.0, 1.3.6, 1.4.11
>Reporter: Sean Busbey
>Assignee: Viraj Jasani
>Priority: Blocker
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-23015.branch-1.3.000.patch
>
>
> HBASE-22728 moved out jackson transitive dependencies. mostly good, but
> moving jackson2 to provided in hbase-server broke few things
> testing-util needs a transitive jackson 2 in order to start the minicluster,
> currently fails with CNFE for {{com.fasterxml.jackson.databind.ObjectMapper}}
> when trying to initialize the master.
> shaded-testing-util needs a relocated jackson 2 for the same reason
> it's not used for any of the mapreduce stuff in hbase-server, so
> {{hbase-shaded-server}} for that purpose should be fine. But it is used by
> {{WALPrettyPrinter}} and some folks might expect that to work from that
> artifact since it is present.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23015) branch-1 hbase-server, testing util, and shaded testing util need jackson
[
https://issues.apache.org/jira/browse/HBASE-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934004#comment-16934004
]
Lars Hofhansl commented on HBASE-23015:
---
I meant apply *locally* above. In any case it does not apply cleanly in my
setup, so it'd be a chunk of work anyway.
> branch-1 hbase-server, testing util, and shaded testing util need jackson
> --
>
> Key: HBASE-23015
> URL: https://issues.apache.org/jira/browse/HBASE-23015
> Project: HBase
> Issue Type: Bug
> Components: Client, shading
>Affects Versions: 1.5.0, 1.3.6, 1.4.11
>Reporter: Sean Busbey
>Assignee: Viraj Jasani
>Priority: Blocker
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-23015.branch-1.3.000.patch
>
>
> HBASE-22728 moved out jackson transitive dependencies. mostly good, but
> moving jackson2 to provided in hbase-server broke few things
> testing-util needs a transitive jackson 2 in order to start the minicluster,
> currently fails with CNFE for {{com.fasterxml.jackson.databind.ObjectMapper}}
> when trying to initialize the master.
> shaded-testing-util needs a relocated jackson 2 for the same reason
> it's not used for any of the mapreduce stuff in hbase-server, so
> {{hbase-shaded-server}} for that purpose should be fine. But it is used by
> {{WALPrettyPrinter}} and some folks might expect that to work from that
> artifact since it is present.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (HBASE-22941) MetaTableAccessor.getMergeRegions() returns parent regions in random order
[ https://issues.apache.org/jira/browse/HBASE-22941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-22941: --- Seems not in branch-2 and branch-2.1? Only in master and branch-2.2... > MetaTableAccessor.getMergeRegions() returns parent regions in random order > -- > > Key: HBASE-22941 > URL: https://issues.apache.org/jira/browse/HBASE-22941 > Project: HBase > Issue Type: Bug >Affects Versions: master >Reporter: István Tóth >Assignee: István Tóth >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.1, 2.1.7 > > > The multi-region merge patch has changed the behaviour of the > MetaTableAccessor.getMergeRegions() method. Earlier, the parent regions were > returned in the same order that they were passed to mergeRegion(). > The patch has changed this to return the regions in random order. To be more > precise, the parent regions are stored in random order in Meta. > I propose storing and returning the parent regions in ascending order. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-22649) Encode StoreFile path URLs in the UI to handle scenarios where CF contains special characters (like # etc.)
[
https://issues.apache.org/jira/browse/HBASE-22649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933995#comment-16933995
]
Hudson commented on HBASE-22649:
Results for branch master
[build #1466 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1466/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1466//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1466//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1466//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Encode StoreFile path URLs in the UI to handle scenarios where CF contains
> special characters (like # etc.)
> ---
>
> Key: HBASE-22649
> URL: https://issues.apache.org/jira/browse/HBASE-22649
> Project: HBase
> Issue Type: Bug
> Components: UI
>Affects Versions: 3.0.0, 2.1.5, 1.3.5
>Reporter: Ashok shetty
>Assignee: Y. SREENIVASULU REDDY
>Priority: Major
> Fix For: 3.0.0, 1.3.6, 2.1.7
>
> Attachments: HBASE-22649.branch-1.002.patch,
> HBASE-22649.branch-1.003.patch, HBASE-22649.branch-1.patch,
> HBASE-22649.branch-2.patch, HBASE-22649.branch-2_002.patch,
> HBASE-22649.patch, HBASE-22649_002.patch
>
>
> 【Test step】：
> 1. create 'specialchar' ,'#'
> 2.put 'specialchar','r1','#:cq','1000'
> 3.flush 'specialchar'
> 4.put 'specialchar','r2','#:cq','1000'
> 5.flush 'specialchar'
>
> Once hfile is created, click the hfile link in UI.
> The following error is throwing.
> {noformat}
> java.io.FileNotFoundException: Path is not a file:
> /hbase/data/default/specialchar/df9d19830c562c4eeb3f8b396211d52d
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:90)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:153)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1942)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:739)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:432)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:878)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:824)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2684)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Reidddddd commented on a change in pull request #615: HBASE-22975 Add read and write QPS metrics at server level and table level
Reidd commented on a change in pull request #615: HBASE-22975 Add read and
write QPS metrics at server level and table level
URL: https://github.com/apache/hbase/pull/615#discussion_r326456309
##
File path:
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableQueryMeterImpl.java
##
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver;
+
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.metrics.Meter;
+import org.apache.hadoop.hbase.metrics.MetricRegistry;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * Implementation of {@link MetricsTableQueryMeter} to track query per second
for one table in
+ * a RegionServer.
+ */
+@InterfaceAudience.Private
+public class MetricsTableQueryMeterImpl implements MetricsTableQueryMeter {
+ private final Map metersByTable = new
ConcurrentHashMap<>();
Review comment:
nit, space between `TableName,TableMeters`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Reidddddd commented on a change in pull request #645: HBASE-22930 Set unique name to longCompactions/shortCompactions threads
Reidd commented on a change in pull request #645: HBASE-22930 Set unique
name to longCompactions/shortCompactions threads
URL: https://github.com/apache/hbase/pull/645#discussion_r326455790
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
##
@@ -131,11 +132,12 @@ public Thread newThread(Runnable r) {
private void createSplitExcecutors() {
final String n = Thread.currentThread().getName();
int splitThreads = conf.getInt(SPLIT_THREADS, SPLIT_THREADS_DEFAULT);
+final AtomicInteger splitThreadCounter = new AtomicInteger(0);
this.splits =
(ThreadPoolExecutor) Executors.newFixedThreadPool(splitThreads, new
ThreadFactory() {
@Override
Review comment:
Only one place not sure: shouldn't the counter be placed in `ThreadFactory`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Comment Edited] (HBASE-23052) hbase-thirdparty version of GSON that works for branch-1
[ https://issues.apache.org/jira/browse/HBASE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933962#comment-16933962 ] Sean Busbey edited comment on HBASE-23052 at 9/20/19 2:31 AM: -- First option: make it so [hbase-thirdparty is generally jdk7 compat PR#3|https://github.com/apache/hbase-thirdparty/pull/3]. I don't like this option because we have to downgrade commons-collection4 and are basically stuck there since that project has dropped jdk7 support. was (Author: busbey): First option: make it so [hbase-thirdparty is generally jdk7 compat PR#3](https://github.com/apache/hbase-thirdparty/pull/3). I don't like this option because we have to downgrade commons-collection4 and are basically stuck there since that project has dropped jdk7 support. > hbase-thirdparty version of GSON that works for branch-1 > > > Key: HBASE-23052 > URL: https://issues.apache.org/jira/browse/HBASE-23052 > Project: HBase > Issue Type: Improvement > Components: dependencies >Affects Versions: thirdparty-2.2.1 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > > HBASE-23015 is buttoning up a needed move off of jackson 1 in branches-1. > We've already got the implementation work in place to move onto the > hbase-thirdparty relocated GSON, but we can't currently build because other > dependencies included in the miscellaneous module is JDK8+ only and branch-1 > needs to work for jdk7. > couple of options: > * make the entire hbase-thirdparty repo work with jdk7 > * break out gson from the clearing house miscellaneous module and make *just* > the new gson module jdk7 compatible > * make a jdk7 compatible miscellaneous module and move gson over there (in > case we decide to move branch-1 off of other problematic libraries e.g. guava) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23052) hbase-thirdparty version of GSON that works for branch-1
[ https://issues.apache.org/jira/browse/HBASE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933962#comment-16933962 ] Sean Busbey commented on HBASE-23052: - First option: make it so [hbase-thirdparty is generally jdk7 compat PR#3](https://github.com/apache/hbase-thirdparty/pull/3). I don't like this option because we have to downgrade commons-collection4 and are basically stuck there since that project has dropped jdk7 support. > hbase-thirdparty version of GSON that works for branch-1 > > > Key: HBASE-23052 > URL: https://issues.apache.org/jira/browse/HBASE-23052 > Project: HBase > Issue Type: Improvement > Components: dependencies >Affects Versions: thirdparty-2.2.1 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > > HBASE-23015 is buttoning up a needed move off of jackson 1 in branches-1. > We've already got the implementation work in place to move onto the > hbase-thirdparty relocated GSON, but we can't currently build because other > dependencies included in the miscellaneous module is JDK8+ only and branch-1 > needs to work for jdk7. > couple of options: > * make the entire hbase-thirdparty repo work with jdk7 > * break out gson from the clearing house miscellaneous module and make *just* > the new gson module jdk7 compatible > * make a jdk7 compatible miscellaneous module and move gson over there (in > case we decide to move branch-1 off of other problematic libraries e.g. guava) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] pankaj72981 opened a new pull request #645: HBASE-22930 Set unique name to longCompactions/shortCompactions threads
pankaj72981 opened a new pull request #645: HBASE-22930 Set unique name to longCompactions/shortCompactions threads URL: https://github.com/apache/hbase/pull/645 Changes for branch-1. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23054) Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm
[
https://issues.apache.org/jira/browse/HBASE-23054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933954#comment-16933954
]
HBase QA commented on HBASE-23054:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
49s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
50s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 5m
31s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
41s{color} | {color:red} hbase-server: The patch generated 2 new + 0 unchanged
- 0 fixed = 2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
16s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
21m 50s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}262m 41s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}339m 10s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated |
| | hadoop.hbase.replication.TestSyncReplicationStandbyKillRS |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.2 Server=19.03.2 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/911/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-23054 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12980778/HBASE-23054.master.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 562e5d714d02 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 0ec0a9403f |
| Default Java |
[GitHub] [hbase] busbey commented on issue #616: HBASE-23015 : Moving from Jackson2 to shaded Gson (Backport HBASE-20587)
busbey commented on issue #616: HBASE-23015 : Moving from Jackson2 to shaded Gson (Backport HBASE-20587) URL: https://github.com/apache/hbase/pull/616#issuecomment-533376069 I have some ideas on moving this forward. discussion on [HBASE-23052](https://issues.apache.org/jira/browse/HBASE-23052) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23015) branch-1 hbase-server, testing util, and shaded testing util need jackson
[
https://issues.apache.org/jira/browse/HBASE-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933952#comment-16933952
]
Sean Busbey commented on HBASE-23015:
-
let me be clear, non released branches are not for down stream. I will revert
this if it is committed before it is ready.
> branch-1 hbase-server, testing util, and shaded testing util need jackson
> --
>
> Key: HBASE-23015
> URL: https://issues.apache.org/jira/browse/HBASE-23015
> Project: HBase
> Issue Type: Bug
> Components: Client, shading
>Affects Versions: 1.5.0, 1.3.6, 1.4.11
>Reporter: Sean Busbey
>Assignee: Viraj Jasani
>Priority: Blocker
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-23015.branch-1.3.000.patch
>
>
> HBASE-22728 moved out jackson transitive dependencies. mostly good, but
> moving jackson2 to provided in hbase-server broke few things
> testing-util needs a transitive jackson 2 in order to start the minicluster,
> currently fails with CNFE for {{com.fasterxml.jackson.databind.ObjectMapper}}
> when trying to initialize the master.
> shaded-testing-util needs a relocated jackson 2 for the same reason
> it's not used for any of the mapreduce stuff in hbase-server, so
> {{hbase-shaded-server}} for that purpose should be fine. But it is used by
> {{WALPrettyPrinter}} and some folks might expect that to work from that
> artifact since it is present.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23015) branch-1 hbase-server, testing util, and shaded testing util need jackson
[
https://issues.apache.org/jira/browse/HBASE-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933949#comment-16933949
]
Sean Busbey commented on HBASE-23015:
-
please see the discussion on the PR. do not merge this. it will break branch-1.
I am actively working on a fix. it is tracked on the linked jira that is set as
a blocker of this jira, HBASE-23052
> branch-1 hbase-server, testing util, and shaded testing util need jackson
> --
>
> Key: HBASE-23015
> URL: https://issues.apache.org/jira/browse/HBASE-23015
> Project: HBase
> Issue Type: Bug
> Components: Client, shading
>Affects Versions: 1.5.0, 1.3.6, 1.4.11
>Reporter: Sean Busbey
>Assignee: Viraj Jasani
>Priority: Blocker
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-23015.branch-1.3.000.patch
>
>
> HBASE-22728 moved out jackson transitive dependencies. mostly good, but
> moving jackson2 to provided in hbase-server broke few things
> testing-util needs a transitive jackson 2 in order to start the minicluster,
> currently fails with CNFE for {{com.fasterxml.jackson.databind.ObjectMapper}}
> when trying to initialize the master.
> shaded-testing-util needs a relocated jackson 2 for the same reason
> it's not used for any of the mapreduce stuff in hbase-server, so
> {{hbase-shaded-server}} for that purpose should be fine. But it is used by
> {{WALPrettyPrinter}} and some folks might expect that to work from that
> artifact since it is present.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] busbey commented on issue #644: HBASE-23055 Alter hbase:meta
busbey commented on issue #644: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/644#issuecomment-533371591 Looks like a mistaken push? Lots of old commits? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23051) Remove unneeded Mockito.mock invocations
[
https://issues.apache.org/jira/browse/HBASE-23051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933910#comment-16933910
]
Hudson commented on HBASE-23051:
Results for branch master
[build #1464 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1464/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1464//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1464//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1464//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove unneeded Mockito.mock invocations
>
>
> Key: HBASE-23051
> URL: https://issues.apache.org/jira/browse/HBASE-23051
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> ErrorProne fails the build using the new Mockito version.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile
> (default-testCompile) on project hbase-server: Compilation failure:
> Compilation failure:
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/MockMasterServices.java:[147,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedureControllers.java:[331,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR] -> [Help 1] {noformat}
> [https://builds.apache.org/job/HBase%20Nightly/job/master/1456/artifact/output-jdk8-hadoop2/patch-compile-root.txt]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23047) ChecksumUtil.validateChecksum logs an INFO message inside a "if(LOG.isTraceEnabled())" block.
[
https://issues.apache.org/jira/browse/HBASE-23047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933861#comment-16933861
]
Hudson commented on HBASE-23047:
Results for branch master
[build #1463 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1463/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1463//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1463//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1463//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> ChecksumUtil.validateChecksum logs an INFO message inside a
> "if(LOG.isTraceEnabled())" block.
> -
>
> Key: HBASE-23047
> URL: https://issues.apache.org/jira/browse/HBASE-23047
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.3.0, 2.2.1, 2.1.6
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-23047.master.001.patch,
> HBASE-23047.master.002.patch
>
>
> Noticed this while analysing another potential checksum issue. Despite doing
> a check for TRACE level, we log an INFO message inside the if block:
> {noformat}
> if (LOG.isTraceEnabled()) {
> LOG.info("dataLength=" + buf.capacity() + ", sizeWithHeader=" +
> onDiskDataSizeWithHeader
> + ", checksumType=" + ctype.getName() + ", file=" + pathName + ",
> offset=" + offset
> + ", headerSize=" + hdrSize + ", bytesPerChecksum=" +
> bytesPerChecksum);
> }
> {noformat}
> Uploading a patch that logs a TRACE message and switch to parameterising
> logging. Since there's no extra computation on the param passing, we
> shouldn't need the extra if either.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-15519) Add per-user metrics
[
https://issues.apache.org/jira/browse/HBASE-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933857#comment-16933857
]
Ankit Singhal commented on HBASE-15519:
---
[~busbey], attaching forward port of the v2 patch and adding lossy counting for
the user metrics. 003.patch is on top of HBASE-23054.
{quote}I think we need testing with more users to see what the load on a RS
looks like. test with > 16 users, > 100 users, > 1000 users. test with long
tail users (like 4 users who are around a lot and then a few hundred who only
show up with freq < metric system refresh)
{quote}
Let me know if we still need scale testing when lossyCounting is implemented.
> Add per-user metrics
> -
>
> Key: HBASE-15519
> URL: https://issues.apache.org/jira/browse/HBASE-15519
> Project: HBase
> Issue Type: Sub-task
> Components: metrics
>Affects Versions: 1.2.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Major
> Attachments: HBASE-15519.master.003.patch, hbase-15519_v0.patch,
> hbase-15519_v1.patch, hbase-15519_v1.patch, hbase-15519_v2.patch
>
>
> Per-user metrics will be useful in multi-tenant cases where we can emit
> number of requests, operations, num RPCs etc at the per-user aggregate level
> per regionserver. We currently have throttles per user, but no way to monitor
> resource usage per-user.
> Looking at these metrics, operators can adjust throttles, do capacity
> planning, etc per-user.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-15519) Add per-user metrics
[ https://issues.apache.org/jira/browse/HBASE-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal updated HBASE-15519: -- Attachment: HBASE-15519.master.003.patch > Add per-user metrics > - > > Key: HBASE-15519 > URL: https://issues.apache.org/jira/browse/HBASE-15519 > Project: HBase > Issue Type: Sub-task > Components: metrics >Affects Versions: 1.2.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar >Priority: Major > Attachments: HBASE-15519.master.003.patch, hbase-15519_v0.patch, > hbase-15519_v1.patch, hbase-15519_v1.patch, hbase-15519_v2.patch > > > Per-user metrics will be useful in multi-tenant cases where we can emit > number of requests, operations, num RPCs etc at the per-user aggregate level > per regionserver. We currently have throttles per user, but no way to monitor > resource usage per-user. > Looking at these metrics, operators can adjust throttles, do capacity > planning, etc per-user. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23015) branch-1 hbase-server, testing util, and shaded testing util need jackson
[
https://issues.apache.org/jira/browse/HBASE-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933851#comment-16933851
]
Lars Hofhansl commented on HBASE-23015:
---
Lemme apply this and see if it fixes the problem I've been seeing.
> branch-1 hbase-server, testing util, and shaded testing util need jackson
> --
>
> Key: HBASE-23015
> URL: https://issues.apache.org/jira/browse/HBASE-23015
> Project: HBase
> Issue Type: Bug
> Components: Client, shading
>Affects Versions: 1.5.0, 1.3.6, 1.4.11
>Reporter: Sean Busbey
>Assignee: Viraj Jasani
>Priority: Blocker
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-23015.branch-1.3.000.patch
>
>
> HBASE-22728 moved out jackson transitive dependencies. mostly good, but
> moving jackson2 to provided in hbase-server broke few things
> testing-util needs a transitive jackson 2 in order to start the minicluster,
> currently fails with CNFE for {{com.fasterxml.jackson.databind.ObjectMapper}}
> when trying to initialize the master.
> shaded-testing-util needs a relocated jackson 2 for the same reason
> it's not used for any of the mapreduce stuff in hbase-server, so
> {{hbase-shaded-server}} for that purpose should be fine. But it is used by
> {{WALPrettyPrinter}} and some folks might expect that to work from that
> artifact since it is present.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack opened a new pull request #644: HBASE-23055 Alter hbase:meta
saintstack opened a new pull request #644: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/644 Make it so hbase:meta can be altered. TableState for hbase:meta is kept in Master. State is in-memory transient so if Master fails, hbase:meta is ENABLED again. hbase:meta schema will be bootstrapped from the filesystem. Changes to filesystem schema are atomic so we should be ok if Master fails mid-edit (TBD) Undoes a bunch of guards that prevented our being able to edit hbase:meta. At minimmum, need to add in a bunch of WARNING. TODO: Tests, more clarity around hbase:meta table state, and undoing references to hard-coded hbase:meta regioninfo. M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java Throw illegal access exception if you try to use MetaTableAccessor getting state of the hbase:meta table. M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java TODO: getTableState needs work in Connection implemetnations. Presumes state is in meta table for all tables. Uses MetaTableAccessor. TODO: More cleanup in here and more cleanup in async versions. M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java Change isTableDisabled/Enabled implementation to ask the Master instead. This will give the Master's TableStateManager's opinion rather than client figuring it for themselves reading meta table direct. M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java TODO: Cleanup in here. Go to master for state, not to meta. M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java Logging cleanup. M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java Shutdown access. M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java Just cleanup. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java Add state holder for hbase:meta. Removed unused methods. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java Shut down access. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23015) branch-1 hbase-server, testing util, and shaded testing util need jackson
[
https://issues.apache.org/jira/browse/HBASE-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933842#comment-16933842
]
Andrew Purtell commented on HBASE-23015:
[~lhofhansl] filed a Phoenix JIRA that indicates they are blocked by this when
trying to use head of branch-1 for testing. I am on vacation without a dev
laptop unfortunately , could someone please commit this?
> branch-1 hbase-server, testing util, and shaded testing util need jackson
> --
>
> Key: HBASE-23015
> URL: https://issues.apache.org/jira/browse/HBASE-23015
> Project: HBase
> Issue Type: Bug
> Components: Client, shading
>Affects Versions: 1.5.0, 1.3.6, 1.4.11
>Reporter: Sean Busbey
>Assignee: Viraj Jasani
>Priority: Blocker
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-23015.branch-1.3.000.patch
>
>
> HBASE-22728 moved out jackson transitive dependencies. mostly good, but
> moving jackson2 to provided in hbase-server broke few things
> testing-util needs a transitive jackson 2 in order to start the minicluster,
> currently fails with CNFE for {{com.fasterxml.jackson.databind.ObjectMapper}}
> when trying to initialize the master.
> shaded-testing-util needs a relocated jackson 2 for the same reason
> it's not used for any of the mapreduce stuff in hbase-server, so
> {{hbase-shaded-server}} for that purpose should be fine. But it is used by
> {{WALPrettyPrinter}} and some folks might expect that to work from that
> artifact since it is present.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22969) A new binary component comparator(BinaryComponentComparator) to perform comparison of arbitrary length and position
[
https://issues.apache.org/jira/browse/HBASE-22969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933835#comment-16933835
]
HBase QA commented on HBASE-22969:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} prototool {color} | {color:blue} 0m
0s{color} | {color:blue} prototool was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
33s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
14s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
2s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
21m 1s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
24s{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
47s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} |
[jira] [Commented] (HBASE-23055) Alter hbase:meta
[ https://issues.apache.org/jira/browse/HBASE-23055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933834#comment-16933834 ] stack commented on HBASE-23055: --- bq. This would be cool, it could also potentially have some benefit to migrations between versions. +1 > Alter hbase:meta > > > Key: HBASE-23055 > URL: https://issues.apache.org/jira/browse/HBASE-23055 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Major > > hbase:meta is currently hardcoded. Its schema cannot be change. > This issue is about allowing edits to hbase:meta schema. It will allow our > being able to set encodings such as the block-with-indexes which will help > quell CPU usage on host carrying hbase:meta. A dynamic hbase:meta is first > step on road to being able to split meta. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23055) Alter hbase:meta
[ https://issues.apache.org/jira/browse/HBASE-23055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933832#comment-16933832 ] Zach York commented on HBASE-23055: --- This would be cool, it could also potentially have some benefit to migrations between versions. > Alter hbase:meta > > > Key: HBASE-23055 > URL: https://issues.apache.org/jira/browse/HBASE-23055 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Major > > hbase:meta is currently hardcoded. Its schema cannot be change. > This issue is about allowing edits to hbase:meta schema. It will allow our > being able to set encodings such as the block-with-indexes which will help > quell CPU usage on host carrying hbase:meta. A dynamic hbase:meta is first > step on road to being able to split meta. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien…
Apache-HBase commented on issue #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien… URL: https://github.com/apache/hbase/pull/361#issuecomment-54919 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 0m 35s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 21s | master passed | | :green_heart: | compile | 1m 27s | master passed | | :green_heart: | checkstyle | 1m 56s | master passed | | :green_heart: | shadedjars | 4m 52s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 1m 2s | master passed | | :blue_heart: | spotbugs | 4m 17s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 5m 26s | master passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 15s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 5m 3s | the patch passed | | :green_heart: | compile | 1m 24s | the patch passed | | :green_heart: | javac | 1m 24s | the patch passed | | :green_heart: | checkstyle | 1m 51s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 43s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 17m 18s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 1m 5s | the patch passed | | :green_heart: | findbugs | 6m 22s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 1m 58s | hbase-client in the patch passed. | | :green_heart: | unit | 165m 40s | hbase-server in the patch passed. | | :green_heart: | asflicense | 1m 2s | The patch does not generate ASF License warnings. | | | | 236m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/361 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 7c01b08952c7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-361/out/precommit/personality/provided.sh | | git revision | master / 1fec3d1dfd | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/2/testReport/ | | Max. process+thread count | 4486 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-23055) Alter hbase:meta
stack created HBASE-23055: - Summary: Alter hbase:meta Key: HBASE-23055 URL: https://issues.apache.org/jira/browse/HBASE-23055 Project: HBase Issue Type: Task Reporter: stack hbase:meta is currently hardcoded. Its schema cannot be change. This issue is about allowing edits to hbase:meta schema. It will allow our being able to set encodings such as the block-with-indexes which will help quell CPU usage on host carrying hbase:meta. A dynamic hbase:meta is first step on road to being able to split meta. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-11288) Splittable Meta
[ https://issues.apache.org/jira/browse/HBASE-11288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-11288. --- Resolution: Incomplete Resolving as incomplete (as Alex makes fun of above). > Splittable Meta > --- > > Key: HBASE-11288 > URL: https://issues.apache.org/jira/browse/HBASE-11288 > Project: HBase > Issue Type: Sub-task >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-18228) HBCK improvements
[ https://issues.apache.org/jira/browse/HBASE-18228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18228: -- Parent: HBASE-22977 Issue Type: Sub-task (was: Improvement) > HBCK improvements > - > > Key: HBASE-18228 > URL: https://issues.apache.org/jira/browse/HBASE-18228 > Project: HBase > Issue Type: Sub-task > Components: hbck >Reporter: Lars Hofhansl >Priority: Critical > Fix For: 3.0.0 > > Attachments: HBASE-18228.branch-1.3.patch > > > We just had a prod issue and running HBCK the way we did actually causes more > problems. > In part HBCK did stuff we did not expect, in part we had little visibility > into what HBCK was doing, and in part the logging was confusing. > I'm proposing 2 improvements: > 1. A dry-run mode. Run, and just list what would have been done. > 2. An interactive mode. Run, and for each action request Y/N user input. So > that a user can opt-out of stuff. > [~jmhsieh], FYI -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] joshelser commented on issue #598: HBASE-22142 Space quota: If table inside namespace having space quota is dropped, data size usage is still considered for the drop table.
joshelser commented on issue #598: HBASE-22142 Space quota: If table inside namespace having space quota is dropped, data size usage is still considered for the drop table. URL: https://github.com/apache/hbase/pull/598#issuecomment-50174 Cool. Looks fine to me. Will try to get this merged in. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-21370) Revamp or remove OfflineMetaRepair
[ https://issues.apache.org/jira/browse/HBASE-21370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-21370. --- Resolution: Duplicate Resolving as duplicate. 5f9fe0c HBASE-22680 [HBCK2] OfflineMetaRepair for hbase2/hbck2 (#6) Above added offlinemetarepair to hbck2. > Revamp or remove OfflineMetaRepair > -- > > Key: HBASE-21370 > URL: https://issues.apache.org/jira/browse/HBASE-21370 > Project: HBase > Issue Type: Task > Components: Operability >Reporter: stack >Priority: Major > > See http://hbase.apache.org/book.html#_special_case_root_and_meta_are_corrupt > It describes the OfflineMetaRepair tool. Does this work in hbase2? This issue > is about removing or fixing it up so it works again. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-20824) IllegalArgumentException: state=WAITING:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS SplitTableRegionProcedure
[
https://issues.apache.org/jira/browse/HBASE-20824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-20824.
---
Resolution: Won't Fix
Resolving old issue about condition we probably fixed long time ago.
> IllegalArgumentException:
> state=WAITING:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS
> SplitTableRegionProcedure
>
>
> Key: HBASE-20824
> URL: https://issues.apache.org/jira/browse/HBASE-20824
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Reporter: stack
>Priority: Major
>
> Running tests to check HBASE-20796 STUCK RIT though region successfully
> assigned, I came across this case.
> Log is throwing out these:
> {code}
> STUCK Region-In-Transition rit=CLOSED,
> location=ve0536.X.Y.Z.com,16020,1530233509248,
> table=IntegrationTestBigLinkedList, region=18643f59d7db3689eeb133062ae4bc4f
> {code}
> Backup up I got this:
> {code}
> 2018-06-28 17:57:32,744 INFO [PEWorker-3]
> procedure.MasterProcedureScheduler: pid=2716,
> state=WAITING:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS;
> SplitTableRegionProcedure table=IntegrationTestBigLinkedList,
> parent=18643f59d7db3689eeb133062ae4bc4f,
> daughterA=6b0d2a0d22113c529dd49fc923a35ad9,
> daughterB=be72b5916a6a11d9ddce7d8af24f129f checking lock on
> 18643f59d7db3689eeb133062ae4bc4f
> 2018-06-28 17:57:32,745 WARN [PEWorker-3] procedure2.ProcedureExecutor:
> Worker terminating UNNATURALLY null
>
> java.lang.IllegalArgumentException:
> pid=2716, state=WAITING:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS;
> SplitTableRegionProcedure table=IntegrationTestBigLinkedList,
> parent=18643f59d7db3689eeb133062ae4bc4f,
> daughterA=6b0d2a0d22113c529dd49fc923a35ad9,
> daughterB=be72b5916a6a11d9ddce7d8af24f129f
>
>
>at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:134)
>
>
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1449)
>
>
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1240)
>
>
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
>
>
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1760)
> {code}
> Just before this I see the below:
> {code}
> 2018-06-28 17:57:32,604 WARN [master/ve0524:16000.Chore.1]
> master.CatalogJanitor: Failed initial scan of catalog table
>
>
> org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
>
>
>at
> org.apache.hadoop.hbase.master.HMaster.isInMaintenanceMode(HMaster.java:2823)
>
>
> at
> org.apache.hadoop.hbase.master.CatalogJanitor.scan(CatalogJanitor.java:281)
>
>
> at
> org.apache.hadoop.hbase.master.CatalogJanitor.initialChore(CatalogJanitor.java:81)
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:184)
>
>
[jira] [Resolved] (HBASE-9270) [0.94] FSTableDescriptors caching is racy
[
https://issues.apache.org/jira/browse/HBASE-9270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-9270.
--
Resolution: Won't Fix
Resolving a 0.94 issue not being worked on.
> [0.94] FSTableDescriptors caching is racy
> -
>
> Key: HBASE-9270
> URL: https://issues.apache.org/jira/browse/HBASE-9270
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.11
>Reporter: Andrew Purtell
>Assignee: Rekha Joshi
>Priority: Minor
> Attachments: HBASE-9270.1.patch
>
>
> An occasionally failing test in 0.92 branch that concurrently executes master
> operations on a single table found this problem in FSTableDescriptors:
> {code}
> diff --git src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> index e882621..b0042cd 100644
> --- src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> +++ src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> @@ -221,8 +221,15 @@ public class FSTableDescriptors implements
> TableDescriptors {
> if
> (HConstants.HBASE_NON_USER_TABLE_DIRS.contains(htd.getNameAsString())) {
>throw new NotImplementedException();
> }
> -if (!this.fsreadonly) updateHTableDescriptor(this.fs, this.rootdir, htd);
> -long modtime = getTableInfoModtime(this.fs, this.rootdir,
> htd.getNameAsString());
> +if (fsreadonly) {
> + // Cannot cache here.
> + // We can't know if a modtime from the most recent file found in a
> + // directory listing at some arbitrary point in time still corresponds
> + // to the latest, nor that our htd is the latest.
> + return;
> +}
> +// Cache with the modtime of the descriptor we wrote
> +long modtime = updateHTableDescriptor(this.fs, this.rootdir,
> htd).getModificationTime();
> this.cache.put(htd.getNameAsString(), new
> TableDescriptorModtime(modtime, htd));
>}
> {code}
> After HBASE-7305 master operations are serialized by a write lock on the
> table.
> However, 0.94 has code with the same issue:
> {code}
> @Override
> public void add(HTableDescriptor htd) throws IOException {
> if (Bytes.equals(HConstants.ROOT_TABLE_NAME, htd.getName())) {
> throw new NotImplementedException();
> }
> if (Bytes.equals(HConstants.META_TABLE_NAME, htd.getName())) {
> throw new NotImplementedException();
> }
> if (HConstants.HBASE_NON_USER_TABLE_DIRS.contains(htd.getNameAsString()))
> {
> throw new NotImplementedException();
> }
> if (!this.fsreadonly) updateHTableDescriptor(this.fs, this.rootdir, htd);
> String tableName = htd.getNameAsString();
> long modtime = getTableInfoModtime(this.fs, this.rootdir, tableName);
> long dirmodtime = getTableDirModtime(this.fs, this.rootdir, tableName);
> this.cache.put(tableName, new TableDescriptorModtime(modtime, dirmodtime,
> htd));
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-16099) Procedures tests timing out for RPC not dispatched or stuck into the queue
[ https://issues.apache.org/jira/browse/HBASE-16099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-16099. --- Resolution: Later Resolving old issue not being worked on as later. > Procedures tests timing out for RPC not dispatched or stuck into the queue > -- > > Key: HBASE-16099 > URL: https://issues.apache.org/jira/browse/HBASE-16099 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Appy >Priority: Major > > Just triaging some tests failures upstream: > https://builds.apache.org/view/All/job/HBase-Trunk_matrix/jdk=latest1.8,label=yahoo-not-h2/1101/testReport/junit/org.apache.hadoop.hbase.master.procedure/TestDeleteColumnFamilyProcedure/testDeleteColumnFamily/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-8687) When moving region with region_mover.rb, there is long stack trace for RegionMovedException
[
https://issues.apache.org/jira/browse/HBASE-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-8687.
--
Resolution: Later
No progress on old issue. Resolving as later.
> When moving region with region_mover.rb, there is long stack trace for
> RegionMovedException
> ---
>
> Key: HBASE-8687
> URL: https://issues.apache.org/jira/browse/HBASE-8687
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Reporter: Ted Yu
>Priority: Minor
>
> When gracefully rolling restart region servers, I saw the following in output:
> {code}
> 2013-06-04 20:44:40,135 DEBUG [main] client.ClientScanner: Scan
> table=usertable, startRow=user8129671889902366092
> 2013-06-04 20:44:40,141 DEBUG [main] client.ClientScanner: Scan table=.META.,
> startRow=usertable,user8129671889902366092,00
> 2013-06-04 20:44:40,158 INFO [main] region_mover: Moving region
> 13168d8b86f1ace9472f60555207a707 (2 of 2) to
> server=hor8n09.gq1.ygridcore.net,60020,1370378675859
> 2013-06-04 20:44:40,405 DEBUG [main] client.ClientScanner: Scan
> table=usertable, startRow=user8129671889902366092
> 2013-06-04 20:44:40,407 WARN [main] client.ServerCallable: Call exception,
> tries=0, numRetries=100
> org.apache.hadoop.hbase.exceptions.RegionMovedException: Region moved to:
> hostname=hor8n09.gq1.ygridcore.net port=60020 startCode=1370378675859. As of
> locationSeqNum=194375.
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:230)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:299)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:147)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:55)
> at
> org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:174)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:215)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:130)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:585)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:450)
> at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:311)
> at
> org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:59)
> at
> org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:167)
> at
> homes.hortonzy.hbase_minus_0_dot_95_dot_1.bin.region_mover.method__6$RUBY$isSuccessfulScan(/homes/hortonzy/hbase-0.95.1/bin/region_mover.rb:121)
> at
> homes$hortonzy$hbase_minus_0_dot_95_dot_1$bin$region_mover$method__6$RUBY$isSuccessfulScan.call(homes$hortonzy$hbase_minus_0_dot_95_dot_1$bin$region_mover$method__6$RUBY$isSuccessfulScan:65535)
> at
> homes$hortonzy$hbase_minus_0_dot_95_dot_1$bin$region_mover$method__6$RUBY$isSuccessfulScan.call(homes$hortonzy$hbase_minus_0_dot_95_dot_1$bin$region_mover$method__6$RUBY$isSuccessfulScan:65535)
> at
> org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:201)
> at
> homes.hortonzy.hbase_minus_0_dot_95_dot_1.bin.region_mover.method__8$RUBY$move(/homes/hortonzy/hbase-0.95.1/bin/region_mover.rb:164)
> at
> homes$hortonzy$hbase_minus_0_dot_95_dot_1$bin$region_mover$method__8$RUBY$move.call(homes$hortonzy$hbase_minus_0_dot_95_dot_1$bin$region_mover$method__8$RUBY$move:65535)
> at
> org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:181)
> at
> org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:69)
> at
> homes.hortonzy.hbase_minus_0_dot_95_dot_1.bin.region_mover.block_6$RUBY$__for__(/homes/hortonzy/hbase-0.95.1/bin/region_mover.rb:381)
> at
>
[jira] [Resolved] (HBASE-20402) [hbck] support verbose option for hbck and by default don't display log messages on stdout/stderr
[
https://issues.apache.org/jira/browse/HBASE-20402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-20402.
---
Resolution: Won't Fix
Work on hbck1. Won't happen.
> [hbck] support verbose option for hbck and by default don't display log
> messages on stdout/stderr
> -
>
> Key: HBASE-20402
> URL: https://issues.apache.org/jira/browse/HBASE-20402
> Project: HBase
> Issue Type: Improvement
>Reporter: Umesh Agashe
>Priority: Major
>
> hbck output by default contains lot of clutter (log messages) which can be
> displayed if -verbose is specified E.g.
> {code}
> $ bin/hbase hbck -summary
> 2018-04-12 13:16:10,695 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> 2018-04-12 13:16:11,027 INFO [main] zookeeper.RecoverableZooKeeper: Process
> identifier=hbase Fsck connecting to ZooKeeper ensemble=localhost:2181
> 2018-04-12 13:16:11,056 INFO [main] zookeeper.ZooKeeper: Client
> environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f,
> built on 03/23/2017 10:13 GMT
> 2018-04-12 13:16:11,056 INFO [main] zookeeper.ZooKeeper: Client
> environment:host.name=dhcp-10-16-0-251.pa.cloudera.com
> 2018-04-12 13:16:11,056 INFO [main] zookeeper.ZooKeeper: Client
> environment:java.version=1.8.0_151
> 2018-04-12 13:16:11,056 INFO [main] zookeeper.ZooKeeper: Client
> environment:java.vendor=Oracle Corporation
> 2018-04-12 13:16:11,056 INFO [main] zookeeper.ZooKeeper: Client
> environment:java.home=/Library/Java/JavaVirtualMachines/jdk1.8.0_151.jdk/Contents/Home/jre
> 2018-04-12 13:16:11,056 INFO [main] zookeeper.ZooKeeper:
> /lib/osgi-resource-locator-1.0.1.jar:/Users/uagashe/dev/apache/hbase-standalone/hbase-3.0.0-SNAPSHOT/bin/../lib/paranamer-2.3.jar:/Users/uagashe/dev/apache/hbase-standalone/hbase-3.0.0-SNAPSHOT/bin/../lib/protobuf-java-2.5.0.jar:/Users/uagashe/dev/apache/hbase-standalone/hbase-3.0.0-SNAPSHOT/bin/../lib/slf4j-api-1.7.25.jar:/Users/uagashe/dev/apache/hbase-standalone/hbase-3.0.0-SNAPSHOT/bin/../lib/slf4j-log4j12-1.7.25.jar:/Users/uagashe/dev/apache/hbase-standalone/hbase-3.0.0-SNAPSHOT/bin/../lib/snappy-java-1.1.4.jar:/Users/uagashe/dev/apache/hbase-standalone/hbase-3.0.0-SNAPSHOT/bin/../lib/spymemcached-2.12.2.jar:/Users/uagashe/dev/apache/hbase-standalone/hbase-3.0.0-SNAPSHOT/bin/../lib/validation-api-1.1.0.Final.jar:/Users/uagashe/dev/apache/hbase-standalone/hbase-3.0.0-SNAPSHOT/bin/../lib/xmlenc-0.52.jar:/Users/uagashe/dev/apache/hbase-standalone/hbase-3.0.0-SNAPSHOT/bin/../lib/xz-1.0.jar:/Users/uagashe/dev/apache/hbase-standalone/hbase-3.0.0-SNAPSHOT/bin/../lib/zookeeper-3.4.10.jar:
> 2018-04-12 13:16:11,056 INFO [main] zookeeper.ZooKeeper: Client
> environment:java.library.path=/Users/uagashe/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
> 2018-04-12 13:16:11,056 INFO [main] zookeeper.ZooKeeper: Client
> environment:java.io.tmpdir=/var/folders/h0/l18445qs64dc1cyb5gcqvyr4gp/T/
> 2018-04-12 13:16:11,056 INFO [main] zookeeper.ZooKeeper: Client
> environment:java.compiler=
> 2018-04-12 13:16:11,056 INFO [main] zookeeper.ZooKeeper: Client
> environment:os.name=Mac OS X
> 2018-04-12 13:16:11,056 INFO [main] zookeeper.ZooKeeper: Client
> environment:os.arch=x86_64
> 2018-04-12 13:16:11,056 INFO [main] zookeeper.ZooKeeper: Client
> environment:os.version=10.13.3
> 2018-04-12 13:16:11,056 INFO [main] zookeeper.ZooKeeper: Client
> environment:user.name=uagashe
> 2018-04-12 13:16:11,056 INFO [main] zookeeper.ZooKeeper: Client
> environment:user.home=/Users/uagashe
> 2018-04-12 13:16:11,056 INFO [main] zookeeper.ZooKeeper: Client
> environment:user.dir=/Users/uagashe/dev/apache/hbase-standalone/hbase-3.0.0-SNAPSHOT
> 2018-04-12 13:16:11,066 INFO [main] zookeeper.ZooKeeper: Initiating client
> connection, connectString=localhost:2181 sessionTimeout=9
> watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@233795b6
> 2018-04-12 13:16:11,091 INFO [main-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL
> (unknown error)
> 2018-04-12 13:16:11,119 INFO [main-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Socket connection established to
> localhost/0:0:0:0:0:0:0:1:2181, initiating session
> 2018-04-12 13:16:11,129 INFO [main-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Session establishment complete on server
> localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x162bb7704d20021, negotiated
> timeout = 9
> 2018-04-12 13:16:11,235 INFO [main] util.HBaseFsck: HBaseFsck Version: 0.0.0
>
[jira] [Commented] (HBASE-23051) Remove unneeded Mockito.mock invocations
[
https://issues.apache.org/jira/browse/HBASE-23051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933801#comment-16933801
]
Hudson commented on HBASE-23051:
Results for branch branch-2.1
[build #1620 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1620/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1620//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1620//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1620//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1620//console].
> Remove unneeded Mockito.mock invocations
>
>
> Key: HBASE-23051
> URL: https://issues.apache.org/jira/browse/HBASE-23051
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> ErrorProne fails the build using the new Mockito version.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile
> (default-testCompile) on project hbase-server: Compilation failure:
> Compilation failure:
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/MockMasterServices.java:[147,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedureControllers.java:[331,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR] -> [Help 1] {noformat}
> [https://builds.apache.org/job/HBase%20Nightly/job/master/1456/artifact/output-jdk8-hadoop2/patch-compile-root.txt]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22969) A new binary component comparator(BinaryComponentComparator) to perform comparison of arbitrary length and position
[
https://issues.apache.org/jira/browse/HBASE-22969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933792#comment-16933792
]
Udai Bhan Kashyap commented on HBASE-22969:
---
[~clayb] - I think I have addressed all your concerns in the latest patch
[^HBASE-22969.0009.patch] . Hopefully, the patch will pass the automated checks!
> A new binary component comparator(BinaryComponentComparator) to perform
> comparison of arbitrary length and position
> ---
>
> Key: HBASE-22969
> URL: https://issues.apache.org/jira/browse/HBASE-22969
> Project: HBase
> Issue Type: Improvement
> Components: Filters
>Reporter: Udai Bhan Kashyap
>Assignee: Udai Bhan Kashyap
>Priority: Minor
> Attachments: HBASE-22969.0003.patch, HBASE-22969.0004.patch,
> HBASE-22969.0005.patch, HBASE-22969.0006.patch, HBASE-22969.0007.patch,
> HBASE-22969.0008.patch, HBASE-22969.0009.patch,
> HBASE-22969.HBASE-22969.0001.patch, HBASE-22969.master.0001.patch
>
>
> Lets say you have composite key: a+b+c+d. And for simplicity assume that
> a,b,c, and d all are 4 byte integers.
> Now, if you want to execute a query which is semantically same to following
> sql:
> {{"SELECT * from table where a=1 and b > 10 and b < 20 and c > 90 and c < 100
> and d=1"}}
> The only choice you have is to do client side filtering. That could be lots
> of unwanted data going through various software components and network.
> Solution:
> We can create a "component" comparator which takes the value of the
> "component" and its relative position in the key to pass the 'Filter'
> subsystem of the server:
> {code}
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ALL);
> int bOffset = 4;
> byte[] b10 = Bytes.toBytes(10);
> Filter b10Filter = new RowFilter(CompareFilter.CompareOp.GREATER,
> new BinaryComponentComparator(b10,bOffset));
> filterList.addFilter(b10Filter);
> byte[] b20 = Bytes.toBytes(20);
> Filter b20Filter = new RowFilter(CompareFilter.CompareOp.LESS,
> new BinaryComponentComparator(b20,bOffset));
> filterList.addFilter(b20Filter);
> int cOffset = 8;
> byte[] c90 = Bytes.toBytes(90);
> Filter c90Filter = new RowFilter(CompareFilter.CompareOp.GREATER,
> new BinaryComponentComparator(c90,cOffset));
> filterList.addFilter(c90Filter);
> byte[] c100 = Bytes.toBytes(100);
> Filter c100Filter = new RowFilter(CompareFilter.CompareOp.LESS,
> new BinaryComponentComparator(c100,cOffset));
> filterList.addFilter(c100Filter);
> in dOffset = 12;
> byte[] d1 = Bytes.toBytes(1);
> Filter dFilter = new RowFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComponentComparator(d1,dOffset));
> filterList.addFilter(dFilter);
> //build start and end key for scan
> int aOffset = 0;
> byte[] startKey = new byte[16]; //key size with four ints
> Bytes.putInt(startKey,aOffset,1); //a=1
> Bytes.putInt(startKey,bOffset,11); //b=11, takes care of b > 10
> Bytes.putInt(startKey,cOffset,91); //c=91,
> Bytes.putInt(startKey,dOffset,1); //d=1,
> byte[] endKey = new byte[16];
> Bytes.putInt(endKey,aOffset,1); //a=1
> Bytes.putInt(endKey,bOffset,20); //b=20, takes care of b < 20
> Bytes.putInt(endKey,cOffset,100); //c=100,
> Bytes.putInt(endKey,dOffset,1); //d=1,
> //setup scan
> Scan scan = new Scan(startKey,endKey);
> scan.setFilter(filterList);
> //The scanner below now should give only desired rows.
> //No client side filtering is required.
> ResultScanner scanner = table.getScanner(scan);
> {code}
> The comparator can be used with any filter which makes use of
> ByteArrayComparable. Most notably it can be used with ValueFilter to filter
> out KV based on partial comparison of 'values' :
> {code}
> byte[] partialValue = Bytes.toBytes("partial_value");
> int partialValueOffset =
> Filter partialValueFilter = new
> ValueFilter(CompareFilter.CompareOp.GREATER,
> new BinaryComponentComparator(partialValue,partialValueOffset));
> {code}
> Which in turn can be combined with RowFilter to create a poweful predicate:
> {code}
> RowFilter rowFilter = new RowFilter(GREATER, new
> BinaryComponentComparator(Bytes.toBytes("a"),1);
> FilterLiost fl = new FilterList
> (MUST_PASS_ALL,rowFilter,partialValueFilter);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-22969) A new binary component comparator(BinaryComponentComparator) to perform comparison of arbitrary length and position
[
https://issues.apache.org/jira/browse/HBASE-22969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Udai Bhan Kashyap updated HBASE-22969:
--
Status: Open (was: Patch Available)
> A new binary component comparator(BinaryComponentComparator) to perform
> comparison of arbitrary length and position
> ---
>
> Key: HBASE-22969
> URL: https://issues.apache.org/jira/browse/HBASE-22969
> Project: HBase
> Issue Type: Improvement
> Components: Filters
>Reporter: Udai Bhan Kashyap
>Assignee: Udai Bhan Kashyap
>Priority: Minor
> Attachments: HBASE-22969.0003.patch, HBASE-22969.0004.patch,
> HBASE-22969.0005.patch, HBASE-22969.0006.patch, HBASE-22969.0007.patch,
> HBASE-22969.0008.patch, HBASE-22969.0009.patch,
> HBASE-22969.HBASE-22969.0001.patch, HBASE-22969.master.0001.patch
>
>
> Lets say you have composite key: a+b+c+d. And for simplicity assume that
> a,b,c, and d all are 4 byte integers.
> Now, if you want to execute a query which is semantically same to following
> sql:
> {{"SELECT * from table where a=1 and b > 10 and b < 20 and c > 90 and c < 100
> and d=1"}}
> The only choice you have is to do client side filtering. That could be lots
> of unwanted data going through various software components and network.
> Solution:
> We can create a "component" comparator which takes the value of the
> "component" and its relative position in the key to pass the 'Filter'
> subsystem of the server:
> {code}
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ALL);
> int bOffset = 4;
> byte[] b10 = Bytes.toBytes(10);
> Filter b10Filter = new RowFilter(CompareFilter.CompareOp.GREATER,
> new BinaryComponentComparator(b10,bOffset));
> filterList.addFilter(b10Filter);
> byte[] b20 = Bytes.toBytes(20);
> Filter b20Filter = new RowFilter(CompareFilter.CompareOp.LESS,
> new BinaryComponentComparator(b20,bOffset));
> filterList.addFilter(b20Filter);
> int cOffset = 8;
> byte[] c90 = Bytes.toBytes(90);
> Filter c90Filter = new RowFilter(CompareFilter.CompareOp.GREATER,
> new BinaryComponentComparator(c90,cOffset));
> filterList.addFilter(c90Filter);
> byte[] c100 = Bytes.toBytes(100);
> Filter c100Filter = new RowFilter(CompareFilter.CompareOp.LESS,
> new BinaryComponentComparator(c100,cOffset));
> filterList.addFilter(c100Filter);
> in dOffset = 12;
> byte[] d1 = Bytes.toBytes(1);
> Filter dFilter = new RowFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComponentComparator(d1,dOffset));
> filterList.addFilter(dFilter);
> //build start and end key for scan
> int aOffset = 0;
> byte[] startKey = new byte[16]; //key size with four ints
> Bytes.putInt(startKey,aOffset,1); //a=1
> Bytes.putInt(startKey,bOffset,11); //b=11, takes care of b > 10
> Bytes.putInt(startKey,cOffset,91); //c=91,
> Bytes.putInt(startKey,dOffset,1); //d=1,
> byte[] endKey = new byte[16];
> Bytes.putInt(endKey,aOffset,1); //a=1
> Bytes.putInt(endKey,bOffset,20); //b=20, takes care of b < 20
> Bytes.putInt(endKey,cOffset,100); //c=100,
> Bytes.putInt(endKey,dOffset,1); //d=1,
> //setup scan
> Scan scan = new Scan(startKey,endKey);
> scan.setFilter(filterList);
> //The scanner below now should give only desired rows.
> //No client side filtering is required.
> ResultScanner scanner = table.getScanner(scan);
> {code}
> The comparator can be used with any filter which makes use of
> ByteArrayComparable. Most notably it can be used with ValueFilter to filter
> out KV based on partial comparison of 'values' :
> {code}
> byte[] partialValue = Bytes.toBytes("partial_value");
> int partialValueOffset =
> Filter partialValueFilter = new
> ValueFilter(CompareFilter.CompareOp.GREATER,
> new BinaryComponentComparator(partialValue,partialValueOffset));
> {code}
> Which in turn can be combined with RowFilter to create a poweful predicate:
> {code}
> RowFilter rowFilter = new RowFilter(GREATER, new
> BinaryComponentComparator(Bytes.toBytes("a"),1);
> FilterLiost fl = new FilterList
> (MUST_PASS_ALL,rowFilter,partialValueFilter);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-22969) A new binary component comparator(BinaryComponentComparator) to perform comparison of arbitrary length and position
[
https://issues.apache.org/jira/browse/HBASE-22969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Udai Bhan Kashyap updated HBASE-22969:
--
Attachment: HBASE-22969.0009.patch
Status: Patch Available (was: Open)
> A new binary component comparator(BinaryComponentComparator) to perform
> comparison of arbitrary length and position
> ---
>
> Key: HBASE-22969
> URL: https://issues.apache.org/jira/browse/HBASE-22969
> Project: HBase
> Issue Type: Improvement
> Components: Filters
>Reporter: Udai Bhan Kashyap
>Assignee: Udai Bhan Kashyap
>Priority: Minor
> Attachments: HBASE-22969.0003.patch, HBASE-22969.0004.patch,
> HBASE-22969.0005.patch, HBASE-22969.0006.patch, HBASE-22969.0007.patch,
> HBASE-22969.0008.patch, HBASE-22969.0009.patch,
> HBASE-22969.HBASE-22969.0001.patch, HBASE-22969.master.0001.patch
>
>
> Lets say you have composite key: a+b+c+d. And for simplicity assume that
> a,b,c, and d all are 4 byte integers.
> Now, if you want to execute a query which is semantically same to following
> sql:
> {{"SELECT * from table where a=1 and b > 10 and b < 20 and c > 90 and c < 100
> and d=1"}}
> The only choice you have is to do client side filtering. That could be lots
> of unwanted data going through various software components and network.
> Solution:
> We can create a "component" comparator which takes the value of the
> "component" and its relative position in the key to pass the 'Filter'
> subsystem of the server:
> {code}
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ALL);
> int bOffset = 4;
> byte[] b10 = Bytes.toBytes(10);
> Filter b10Filter = new RowFilter(CompareFilter.CompareOp.GREATER,
> new BinaryComponentComparator(b10,bOffset));
> filterList.addFilter(b10Filter);
> byte[] b20 = Bytes.toBytes(20);
> Filter b20Filter = new RowFilter(CompareFilter.CompareOp.LESS,
> new BinaryComponentComparator(b20,bOffset));
> filterList.addFilter(b20Filter);
> int cOffset = 8;
> byte[] c90 = Bytes.toBytes(90);
> Filter c90Filter = new RowFilter(CompareFilter.CompareOp.GREATER,
> new BinaryComponentComparator(c90,cOffset));
> filterList.addFilter(c90Filter);
> byte[] c100 = Bytes.toBytes(100);
> Filter c100Filter = new RowFilter(CompareFilter.CompareOp.LESS,
> new BinaryComponentComparator(c100,cOffset));
> filterList.addFilter(c100Filter);
> in dOffset = 12;
> byte[] d1 = Bytes.toBytes(1);
> Filter dFilter = new RowFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComponentComparator(d1,dOffset));
> filterList.addFilter(dFilter);
> //build start and end key for scan
> int aOffset = 0;
> byte[] startKey = new byte[16]; //key size with four ints
> Bytes.putInt(startKey,aOffset,1); //a=1
> Bytes.putInt(startKey,bOffset,11); //b=11, takes care of b > 10
> Bytes.putInt(startKey,cOffset,91); //c=91,
> Bytes.putInt(startKey,dOffset,1); //d=1,
> byte[] endKey = new byte[16];
> Bytes.putInt(endKey,aOffset,1); //a=1
> Bytes.putInt(endKey,bOffset,20); //b=20, takes care of b < 20
> Bytes.putInt(endKey,cOffset,100); //c=100,
> Bytes.putInt(endKey,dOffset,1); //d=1,
> //setup scan
> Scan scan = new Scan(startKey,endKey);
> scan.setFilter(filterList);
> //The scanner below now should give only desired rows.
> //No client side filtering is required.
> ResultScanner scanner = table.getScanner(scan);
> {code}
> The comparator can be used with any filter which makes use of
> ByteArrayComparable. Most notably it can be used with ValueFilter to filter
> out KV based on partial comparison of 'values' :
> {code}
> byte[] partialValue = Bytes.toBytes("partial_value");
> int partialValueOffset =
> Filter partialValueFilter = new
> ValueFilter(CompareFilter.CompareOp.GREATER,
> new BinaryComponentComparator(partialValue,partialValueOffset));
> {code}
> Which in turn can be combined with RowFilter to create a poweful predicate:
> {code}
> RowFilter rowFilter = new RowFilter(GREATER, new
> BinaryComponentComparator(Bytes.toBytes("a"),1);
> FilterLiost fl = new FilterList
> (MUST_PASS_ALL,rowFilter,partialValueFilter);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22927) Upgrade mockito version for Java 11 compatibility
[
https://issues.apache.org/jira/browse/HBASE-22927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933780#comment-16933780
]
Hudson commented on HBASE-22927:
Results for branch branch-2
[build #2282 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2282/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2282//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2282//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2282//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 3. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2282//artifact/output-integration/hadoop-3.log].
(note that this means we didn't check the Hadoop 3 shaded client)
> Upgrade mockito version for Java 11 compatibility
> -
>
> Key: HBASE-22927
> URL: https://issues.apache.org/jira/browse/HBASE-22927
> Project: HBase
> Issue Type: Sub-task
>Reporter: Sakthi
>Assignee: Rabi Kumar K C
>Priority: Major
> Labels: jdk11
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> Pasting the discussion from HBASE-22534 here:
> "Currently mockito-core version is at 2.1.0. According to
> [https://github.com/mockito/mockito/blob/release/2.x/doc/release-notes/official.md],
> looks like Java 11 compatibility was introduced in 2.19+. And 2.23.2 claims
> to have full java 11 support after byte-buddy fix etc."
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22927) Upgrade mockito version for Java 11 compatibility
[
https://issues.apache.org/jira/browse/HBASE-22927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933782#comment-16933782
]
Hudson commented on HBASE-22927:
Results for branch branch-2.1
[build #1619 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1619/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1619//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1619//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1619//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Upgrade mockito version for Java 11 compatibility
> -
>
> Key: HBASE-22927
> URL: https://issues.apache.org/jira/browse/HBASE-22927
> Project: HBase
> Issue Type: Sub-task
>Reporter: Sakthi
>Assignee: Rabi Kumar K C
>Priority: Major
> Labels: jdk11
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> Pasting the discussion from HBASE-22534 here:
> "Currently mockito-core version is at 2.1.0. According to
> [https://github.com/mockito/mockito/blob/release/2.x/doc/release-notes/official.md],
> looks like Java 11 compatibility was introduced in 2.19+. And 2.23.2 claims
> to have full java 11 support after byte-buddy fix etc."
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23049) TableDescriptors#getAll should return the tables ordering by the name which contain namespace
[
https://issues.apache.org/jira/browse/HBASE-23049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933779#comment-16933779
]
Hudson commented on HBASE-23049:
Results for branch branch-2
[build #2282 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2282/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2282//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2282//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2282//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 3. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2282//artifact/output-integration/hadoop-3.log].
(note that this means we didn't check the Hadoop 3 shaded client)
> TableDescriptors#getAll should return the tables ordering by the name which
> contain namespace
> -
>
> Key: HBASE-23049
> URL: https://issues.apache.org/jira/browse/HBASE-23049
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> TableDescriptors#getAll return a TreeMap ordering by
> TableName#getNameAsString. But if the namespace is "default",
> TableName#getNameAsString just returns the name which not contain the
> namespace "default". Should use TableName#getNameWithNamespaceInclAsString.
> It will effect the tables order in Tables UI and shell "list" result.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23051) Remove unneeded Mockito.mock invocations
[
https://issues.apache.org/jira/browse/HBASE-23051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933772#comment-16933772
]
Hudson commented on HBASE-23051:
Results for branch branch-2
[build #2284 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2284/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2284//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2284//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2284//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2284//console].
> Remove unneeded Mockito.mock invocations
>
>
> Key: HBASE-23051
> URL: https://issues.apache.org/jira/browse/HBASE-23051
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> ErrorProne fails the build using the new Mockito version.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile
> (default-testCompile) on project hbase-server: Compilation failure:
> Compilation failure:
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/MockMasterServices.java:[147,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedureControllers.java:[331,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR] -> [Help 1] {noformat}
> [https://builds.apache.org/job/HBase%20Nightly/job/master/1456/artifact/output-jdk8-hadoop2/patch-compile-root.txt]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-22649) Encode StoreFile path URLs in the UI to handle scenarios where CF contains special characters (like # etc.)
[
https://issues.apache.org/jira/browse/HBASE-22649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sakthi updated HBASE-22649:
---
Summary: Encode StoreFile path URLs in the UI to handle scenarios where CF
contains special characters (like # etc.) (was: FileNotFoundException shown in
UI when tried to access HFILE URL of a column family name have special char
(e.g #))
> Encode StoreFile path URLs in the UI to handle scenarios where CF contains
> special characters (like # etc.)
> ---
>
> Key: HBASE-22649
> URL: https://issues.apache.org/jira/browse/HBASE-22649
> Project: HBase
> Issue Type: Bug
> Components: UI
>Affects Versions: 3.0.0, 2.1.5, 1.3.5
>Reporter: Ashok shetty
>Assignee: Y. SREENIVASULU REDDY
>Priority: Major
> Fix For: 3.0.0, 1.3.6, 2.1.7
>
> Attachments: HBASE-22649.branch-1.002.patch,
> HBASE-22649.branch-1.003.patch, HBASE-22649.branch-1.patch,
> HBASE-22649.branch-2.patch, HBASE-22649.branch-2_002.patch,
> HBASE-22649.patch, HBASE-22649_002.patch
>
>
> 【Test step】：
> 1. create 'specialchar' ,'#'
> 2.put 'specialchar','r1','#:cq','1000'
> 3.flush 'specialchar'
> 4.put 'specialchar','r2','#:cq','1000'
> 5.flush 'specialchar'
>
> Once hfile is created, click the hfile link in UI.
> The following error is throwing.
> {noformat}
> java.io.FileNotFoundException: Path is not a file:
> /hbase/data/default/specialchar/df9d19830c562c4eeb3f8b396211d52d
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:90)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:153)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1942)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:739)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:432)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:878)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:824)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2684)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22649) FileNotFoundException shown in UI when tried to access HFILE URL of a column family name have special char (e.g #)
[
https://issues.apache.org/jira/browse/HBASE-22649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933759#comment-16933759
]
Sakthi commented on HBASE-22649:
Verified the patch, checked the UI. I think it's okay if we skip the UT for
this.
{code:java}
BEFORE:
URL:
http://192.xxx.x.xx:16030/storeFile.jsp?name=file:/path/to/hbase/data/default/specialchar/d2b7b16c188b5dced97121f61d9067a1/#/91aa57d2c59c4257838aec73e9aa9037
UI:
java.io.FileNotFoundException:
/path/to/hbase/data/default/specialchar/d2b7b16c188b5dced97121f61d9067a1 (Is a
directory){code}
{code:java}
AFTER:
URL:
http://192.168.0.69:16030/storeFile.jsp?name=file%3A%2FPath%2Fto%2Fhbase%2Fdata%2Fdefault%2Fspecialchar%2Fd2b7b16c188b5dced97121f61d9067a1%2F%23%2F91aa57d2c59c4257838aec73e9aa9037
UI:
Stats: ...{code}
Let me get this in [~sreenivasulureddy]. Thanks.
> FileNotFoundException shown in UI when tried to access HFILE URL of a column
> family name have special char (e.g #)
> --
>
> Key: HBASE-22649
> URL: https://issues.apache.org/jira/browse/HBASE-22649
> Project: HBase
> Issue Type: Bug
> Components: UI
>Affects Versions: 3.0.0, 2.1.5, 1.3.5
>Reporter: Ashok shetty
>Assignee: Y. SREENIVASULU REDDY
>Priority: Major
> Fix For: 3.0.0, 1.3.6, 2.1.7
>
> Attachments: HBASE-22649.branch-1.002.patch,
> HBASE-22649.branch-1.003.patch, HBASE-22649.branch-1.patch,
> HBASE-22649.branch-2.patch, HBASE-22649.branch-2_002.patch,
> HBASE-22649.patch, HBASE-22649_002.patch
>
>
> 【Test step】：
> 1. create 'specialchar' ,'#'
> 2.put 'specialchar','r1','#:cq','1000'
> 3.flush 'specialchar'
> 4.put 'specialchar','r2','#:cq','1000'
> 5.flush 'specialchar'
>
> Once hfile is created, click the hfile link in UI.
> The following error is throwing.
> {noformat}
> java.io.FileNotFoundException: Path is not a file:
> /hbase/data/default/specialchar/df9d19830c562c4eeb3f8b396211d52d
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:90)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:153)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1942)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:739)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:432)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:878)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:824)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2684)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HBASE-22649) FileNotFoundException shown in UI when tried to access HFILE URL of a column family name have special char (e.g #)
[
https://issues.apache.org/jira/browse/HBASE-22649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933759#comment-16933759
]
Sakthi edited comment on HBASE-22649 at 9/19/19 9:02 PM:
-
Verified the patch, checked the UI. I think it's okay if we skip the UT for
this.
{code:java}
BEFORE:
URL:
http://192.xxx.x.xx:16030/storeFile.jsp?name=file:/path/to/hbase/data/default/specialchar/d2b7b16c188b5dced97121f61d9067a1/#/91aa57d2c59c4257838aec73e9aa9037
UI:
java.io.FileNotFoundException:
/path/to/hbase/data/default/specialchar/d2b7b16c188b5dced97121f61d9067a1 (Is a
directory){code}
{code:java}
AFTER:
URL:
http://192.xxx.x.xx:16030/storeFile.jsp?name=file%3A%2FPath%2Fto%2Fhbase%2Fdata%2Fdefault%2Fspecialchar%2Fd2b7b16c188b5dced97121f61d9067a1%2F%23%2F91aa57d2c59c4257838aec73e9aa9037
UI:
Stats: ...{code}
Let me get this in [~sreenivasulureddy]. Thanks.
was (Author: jatsakthi):
Verified the patch, checked the UI. I think it's okay if we skip the UT for
this.
{code:java}
BEFORE:
URL:
http://192.xxx.x.xx:16030/storeFile.jsp?name=file:/path/to/hbase/data/default/specialchar/d2b7b16c188b5dced97121f61d9067a1/#/91aa57d2c59c4257838aec73e9aa9037
UI:
java.io.FileNotFoundException:
/path/to/hbase/data/default/specialchar/d2b7b16c188b5dced97121f61d9067a1 (Is a
directory){code}
{code:java}
AFTER:
URL:
http://192.168.0.69:16030/storeFile.jsp?name=file%3A%2FPath%2Fto%2Fhbase%2Fdata%2Fdefault%2Fspecialchar%2Fd2b7b16c188b5dced97121f61d9067a1%2F%23%2F91aa57d2c59c4257838aec73e9aa9037
UI:
Stats: ...{code}
Let me get this in [~sreenivasulureddy]. Thanks.
> FileNotFoundException shown in UI when tried to access HFILE URL of a column
> family name have special char (e.g #)
> --
>
> Key: HBASE-22649
> URL: https://issues.apache.org/jira/browse/HBASE-22649
> Project: HBase
> Issue Type: Bug
> Components: UI
>Affects Versions: 3.0.0, 2.1.5, 1.3.5
>Reporter: Ashok shetty
>Assignee: Y. SREENIVASULU REDDY
>Priority: Major
> Fix For: 3.0.0, 1.3.6, 2.1.7
>
> Attachments: HBASE-22649.branch-1.002.patch,
> HBASE-22649.branch-1.003.patch, HBASE-22649.branch-1.patch,
> HBASE-22649.branch-2.patch, HBASE-22649.branch-2_002.patch,
> HBASE-22649.patch, HBASE-22649_002.patch
>
>
> 【Test step】：
> 1. create 'specialchar' ,'#'
> 2.put 'specialchar','r1','#:cq','1000'
> 3.flush 'specialchar'
> 4.put 'specialchar','r2','#:cq','1000'
> 5.flush 'specialchar'
>
> Once hfile is created, click the hfile link in UI.
> The following error is throwing.
> {noformat}
> java.io.FileNotFoundException: Path is not a file:
> /hbase/data/default/specialchar/df9d19830c562c4eeb3f8b396211d52d
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:90)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:153)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1942)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:739)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:432)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:878)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:824)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2684)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23051) Remove unneeded Mockito.mock invocations
[
https://issues.apache.org/jira/browse/HBASE-23051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933758#comment-16933758
]
Hudson commented on HBASE-23051:
Results for branch branch-2.2
[build #625 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/625/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/625//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/625//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/625//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/625//console].
> Remove unneeded Mockito.mock invocations
>
>
> Key: HBASE-23051
> URL: https://issues.apache.org/jira/browse/HBASE-23051
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> ErrorProne fails the build using the new Mockito version.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile
> (default-testCompile) on project hbase-server: Compilation failure:
> Compilation failure:
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/MockMasterServices.java:[147,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedureControllers.java:[331,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR] -> [Help 1] {noformat}
> [https://builds.apache.org/job/HBase%20Nightly/job/master/1456/artifact/output-jdk8-hadoop2/patch-compile-root.txt]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22927) Upgrade mockito version for Java 11 compatibility
[
https://issues.apache.org/jira/browse/HBASE-22927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933747#comment-16933747
]
Hudson commented on HBASE-22927:
Results for branch master
[build #1462 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1462/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1462//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1462//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1462//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Upgrade mockito version for Java 11 compatibility
> -
>
> Key: HBASE-22927
> URL: https://issues.apache.org/jira/browse/HBASE-22927
> Project: HBase
> Issue Type: Sub-task
>Reporter: Sakthi
>Assignee: Rabi Kumar K C
>Priority: Major
> Labels: jdk11
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> Pasting the discussion from HBASE-22534 here:
> "Currently mockito-core version is at 2.1.0. According to
> [https://github.com/mockito/mockito/blob/release/2.x/doc/release-notes/official.md],
> looks like Java 11 compatibility was introduced in 2.19+. And 2.23.2 claims
> to have full java 11 support after byte-buddy fix etc."
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23054) Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm
[ https://issues.apache.org/jira/browse/HBASE-23054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal updated HBASE-23054: -- Description: While trying to use LossyCounting for HBASE-15519 , found following bugs in current implementation – Remove synchronization block from MetaTableMetrics to avoid congestion at the code – Fix license format – Fix LossyCounting algorithm as per [http://www.vldb.org/conf/2002/S10P03.pdf |http://www.vldb.org/conf/2002/S10P03.pdf] -- Avoid doing sweep on every insert in LossyCounting – Remove extra redundant data structures from MetaTableMetrics. was: While trying to use LossyCounting for HBASE-15519 , found following bugs in current implementation – Remove synchronization block from MetaTableMetrics to avoid congestion at the code – Fix license format – Fix LossyCounting algorithm as per [http://www.vldb.org/conf/2002/S10P03.pdf] – Remove extra redundant data structures from MetaTableMetrics. > Remove synchronization block from MetaTableMetrics and fix LossyCounting > algorithm > -- > > Key: HBASE-23054 > URL: https://issues.apache.org/jira/browse/HBASE-23054 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.5 >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Attachments: HBASE-23054.master.001.patch > > > While trying to use LossyCounting for HBASE-15519 , found following bugs in > current implementation > – Remove synchronization block from MetaTableMetrics to avoid congestion at > the code > – Fix license format > – Fix LossyCounting algorithm as per > [http://www.vldb.org/conf/2002/S10P03.pdf > |http://www.vldb.org/conf/2002/S10P03.pdf] > -- Avoid doing sweep on every insert in LossyCounting > – Remove extra redundant data structures from MetaTableMetrics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23054) Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm
[ https://issues.apache.org/jira/browse/HBASE-23054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal updated HBASE-23054: -- Affects Version/s: 2.1.5 Status: Patch Available (was: Open) > Remove synchronization block from MetaTableMetrics and fix LossyCounting > algorithm > -- > > Key: HBASE-23054 > URL: https://issues.apache.org/jira/browse/HBASE-23054 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.5 >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Attachments: HBASE-23054.master.001.patch > > > While trying to use LossyCounting for HBASE-15519 , found following bugs in > current implementation > – Remove synchronization block from MetaTableMetrics to avoid congestion at > the code > – Fix license format > – Fix LossyCounting algorithm as per > [http://www.vldb.org/conf/2002/S10P03.pdf] > – Remove extra redundant data structures from MetaTableMetrics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23054) Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm
[ https://issues.apache.org/jira/browse/HBASE-23054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933743#comment-16933743 ] Ankit Singhal commented on HBASE-23054: --- FYI [~busbey]/[~jatsakthi]/[~xucang] > Remove synchronization block from MetaTableMetrics and fix LossyCounting > algorithm > -- > > Key: HBASE-23054 > URL: https://issues.apache.org/jira/browse/HBASE-23054 > Project: HBase > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Attachments: HBASE-23054.master.001.patch > > > While trying to use LossyCounting for HBASE-15519 , found following bugs in > current implementation > – Remove synchronization block from MetaTableMetrics to avoid congestion at > the code > – Fix license format > – Fix LossyCounting algorithm as per > [http://www.vldb.org/conf/2002/S10P03.pdf] > – Remove extra redundant data structures from MetaTableMetrics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23054) Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm
[ https://issues.apache.org/jira/browse/HBASE-23054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal updated HBASE-23054: -- Attachment: HBASE-23054.master.001.patch > Remove synchronization block from MetaTableMetrics and fix LossyCounting > algorithm > -- > > Key: HBASE-23054 > URL: https://issues.apache.org/jira/browse/HBASE-23054 > Project: HBase > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Attachments: HBASE-23054.master.001.patch > > > While trying to use LossyCounting for HBASE-15519 , found following bugs in > current implementation > – Remove synchronization block from MetaTableMetrics to avoid congestion at > the code > – Fix license format > – Fix LossyCounting algorithm as per > [http://www.vldb.org/conf/2002/S10P03.pdf] > – Remove extra redundant data structures from MetaTableMetrics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23054) Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm
[ https://issues.apache.org/jira/browse/HBASE-23054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal updated HBASE-23054: -- Description: While trying to use LossyCounting for HBASE-15519 , found following bugs in current implementation – Remove synchronization block from MetaTableMetrics to avoid congestion at the code – Fix license format – Fix LossyCounting algorithm as per [http://www.vldb.org/conf/2002/S10P03.pdf] – Remove extra redundant data structures from MetaTableMetrics. was: While trying to use LossyCounting for HBASE-15519 , found following bugs -- Remove synchronization block from MetaTableMetrics to avoid congestion at the code -- Fix license format -- Fix LossyCounting algorithm as per [http://www.vldb.org/conf/2002/S10P03.pdf] -- Remove extra redundant data structures from MetaTableMetrics. > Remove synchronization block from MetaTableMetrics and fix LossyCounting > algorithm > -- > > Key: HBASE-23054 > URL: https://issues.apache.org/jira/browse/HBASE-23054 > Project: HBase > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > > While trying to use LossyCounting for HBASE-15519 , found following bugs in > current implementation > – Remove synchronization block from MetaTableMetrics to avoid congestion at > the code > – Fix license format > – Fix LossyCounting algorithm as per > [http://www.vldb.org/conf/2002/S10P03.pdf] > – Remove extra redundant data structures from MetaTableMetrics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23054) Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm
Ankit Singhal created HBASE-23054: - Summary: Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm Key: HBASE-23054 URL: https://issues.apache.org/jira/browse/HBASE-23054 Project: HBase Issue Type: Bug Reporter: Ankit Singhal Assignee: Ankit Singhal While trying to use LossyCounting for HBASE-15519 , found following bugs -- Remove synchronization block from MetaTableMetrics to avoid congestion at the code -- Fix license format -- Fix LossyCounting algorithm as per [http://www.vldb.org/conf/2002/S10P03.pdf] -- Remove extra redundant data structures from MetaTableMetrics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-22927) Upgrade mockito version for Java 11 compatibility
[
https://issues.apache.org/jira/browse/HBASE-22927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933740#comment-16933740
]
Hudson commented on HBASE-22927:
Results for branch branch-2.2
[build #624 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/624/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/624//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/624//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/624//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Upgrade mockito version for Java 11 compatibility
> -
>
> Key: HBASE-22927
> URL: https://issues.apache.org/jira/browse/HBASE-22927
> Project: HBase
> Issue Type: Sub-task
>Reporter: Sakthi
>Assignee: Rabi Kumar K C
>Priority: Major
> Labels: jdk11
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> Pasting the discussion from HBASE-22534 here:
> "Currently mockito-core version is at 2.1.0. According to
> [https://github.com/mockito/mockito/blob/release/2.x/doc/release-notes/official.md],
> looks like Java 11 compatibility was introduced in 2.19+. And 2.23.2 claims
> to have full java 11 support after byte-buddy fix etc."
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22027) Move non-MR parts of TokenUtil into hbase-client
[
https://issues.apache.org/jira/browse/HBASE-22027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933728#comment-16933728
]
Sean Busbey commented on HBASE-22027:
-
please only in new minor releases (e.g. 2.3.0, 1.5.0). it'll be surprising for
a new way of getting tokens to show up in a maintenance release.
(Also as a side comment to folks who land on this issue while upgrading to
HBase 2, TokenUtil should exist in {{hbase-shaded-mapreduce}} as of HBase 2.0.0)
> Move non-MR parts of TokenUtil into hbase-client
>
>
> Key: HBASE-22027
> URL: https://issues.apache.org/jira/browse/HBASE-22027
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.1.3
>Reporter: Stig Rohde Døssing
>Assignee: Stig Rohde Døssing
>Priority: Major
> Attachments:
> 0001-HBase-22027-Split-non-MR-related-parts-of-TokenUtil-.patch,
> 0001-HBase-22027-Split-non-MR-related-parts-of-TokenUtil-.patch,
> 0002-HBase-22027-Split-non-MR-related-parts-of-TokenUtil-.patch
>
>
> HBASE-14208 moved TokenUtil from hbase-client to hbase-server.
> I have a project depending on hbase-client 1.4.4, which I'd like to upgrade
> to 2.1.3. My project uses TokenUtil (specifically obtainAndCacheToken), which
> is included in hbase-client 1.4.4. At the same time I also have a dependency
> on Jetty 9.4, which is incompatible with the current version used by Hadoop.
> I can fix this for hbase-client by using hbase-shaded-client instead, since
> Jetty is shaded in this jar, but TokenUtil is only present in hbase-server as
> of 2.0.0. Since there is no hbase-shaded-server, I can't use TokenUtil and
> Jetty 9.4 at the same time.
> TokenUtil can be split into server-only parts, and a client relevant part
> that can go back to hbase-client. The TokenUtil in hbase-server can retain
> the moved methods, and delegate to the util in hbase-client if backward
> compatibility is a concern.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22027) Move non-MR parts of TokenUtil into hbase-client
[ https://issues.apache.org/jira/browse/HBASE-22027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933720#comment-16933720 ] stack commented on HBASE-22027: --- [~Srdo] I merged your nice patch on master branch. I tried to backport but there are a bunch of differences. Mind putting up a new PR for branch-2? How far back do you want it to go? Thanks for persisting here. > Move non-MR parts of TokenUtil into hbase-client > > > Key: HBASE-22027 > URL: https://issues.apache.org/jira/browse/HBASE-22027 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.1.3 >Reporter: Stig Rohde Døssing >Assignee: Stig Rohde Døssing >Priority: Major > Attachments: > 0001-HBase-22027-Split-non-MR-related-parts-of-TokenUtil-.patch, > 0001-HBase-22027-Split-non-MR-related-parts-of-TokenUtil-.patch, > 0002-HBase-22027-Split-non-MR-related-parts-of-TokenUtil-.patch > > > HBASE-14208 moved TokenUtil from hbase-client to hbase-server. > I have a project depending on hbase-client 1.4.4, which I'd like to upgrade > to 2.1.3. My project uses TokenUtil (specifically obtainAndCacheToken), which > is included in hbase-client 1.4.4. At the same time I also have a dependency > on Jetty 9.4, which is incompatible with the current version used by Hadoop. > I can fix this for hbase-client by using hbase-shaded-client instead, since > Jetty is shaded in this jar, but TokenUtil is only present in hbase-server as > of 2.0.0. Since there is no hbase-shaded-server, I can't use TokenUtil and > Jetty 9.4 at the same time. > TokenUtil can be split into server-only parts, and a client relevant part > that can go back to hbase-client. The TokenUtil in hbase-server can retain > the moved methods, and delegate to the util in hbase-client if backward > compatibility is a concern. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23049) TableDescriptors#getAll should return the tables ordering by the name which contain namespace
[
https://issues.apache.org/jira/browse/HBASE-23049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933719#comment-16933719
]
Hudson commented on HBASE-23049:
Results for branch branch-2.1
[build #1618 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1618/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1618//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1618//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1618//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1618//console].
> TableDescriptors#getAll should return the tables ordering by the name which
> contain namespace
> -
>
> Key: HBASE-23049
> URL: https://issues.apache.org/jira/browse/HBASE-23049
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> TableDescriptors#getAll return a TreeMap ordering by
> TableName#getNameAsString. But if the namespace is "default",
> TableName#getNameAsString just returns the name which not contain the
> namespace "default". Should use TableName#getNameWithNamespaceInclAsString.
> It will effect the tables order in Tables UI and shell "list" result.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack merged pull request #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien…
saintstack merged pull request #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien… URL: https://github.com/apache/hbase/pull/361 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23049) TableDescriptors#getAll should return the tables ordering by the name which contain namespace
[
https://issues.apache.org/jira/browse/HBASE-23049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933703#comment-16933703
]
Hudson commented on HBASE-23049:
Results for branch branch-2.2
[build #623 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/623/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/623//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/623//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/623//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/623//console].
> TableDescriptors#getAll should return the tables ordering by the name which
> contain namespace
> -
>
> Key: HBASE-23049
> URL: https://issues.apache.org/jira/browse/HBASE-23049
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> TableDescriptors#getAll return a TreeMap ordering by
> TableName#getNameAsString. But if the namespace is "default",
> TableName#getNameAsString just returns the name which not contain the
> namespace "default". Should use TableName#getNameWithNamespaceInclAsString.
> It will effect the tables order in Tables UI and shell "list" result.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23047) ChecksumUtil.validateChecksum logs an INFO message inside a "if(LOG.isTraceEnabled())" block.
[
https://issues.apache.org/jira/browse/HBASE-23047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933694#comment-16933694
]
Hudson commented on HBASE-23047:
Results for branch branch-2
[build #2283 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2283/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2283//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2283//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2283//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> ChecksumUtil.validateChecksum logs an INFO message inside a
> "if(LOG.isTraceEnabled())" block.
> -
>
> Key: HBASE-23047
> URL: https://issues.apache.org/jira/browse/HBASE-23047
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.3.0, 2.2.1, 2.1.6
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-23047.master.001.patch,
> HBASE-23047.master.002.patch
>
>
> Noticed this while analysing another potential checksum issue. Despite doing
> a check for TRACE level, we log an INFO message inside the if block:
> {noformat}
> if (LOG.isTraceEnabled()) {
> LOG.info("dataLength=" + buf.capacity() + ", sizeWithHeader=" +
> onDiskDataSizeWithHeader
> + ", checksumType=" + ctype.getName() + ", file=" + pathName + ",
> offset=" + offset
> + ", headerSize=" + hdrSize + ", bytesPerChecksum=" +
> bytesPerChecksum);
> }
> {noformat}
> Uploading a patch that logs a TRACE message and switch to parameterising
> logging. Since there's no extra computation on the param passing, we
> shouldn't need the extra if either.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #615: HBASE-22975 Add read and write QPS metrics at server level and table level
Apache-HBase commented on issue #615: HBASE-22975 Add read and write QPS metrics at server level and table level URL: https://github.com/apache/hbase/pull/615#issuecomment-533246372 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 1m 21s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :yellow_heart: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 0m 35s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 18s | master passed | | :green_heart: | compile | 1m 36s | master passed | | :green_heart: | checkstyle | 1m 53s | master passed | | :green_heart: | shadedjars | 4m 49s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 1m 10s | master passed | | :blue_heart: | spotbugs | 4m 10s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 5m 8s | master passed | | :yellow_heart: | patch | 4m 32s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 15s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 5m 14s | the patch passed | | :green_heart: | compile | 1m 37s | the patch passed | | :green_heart: | javac | 1m 37s | the patch passed | | :green_heart: | checkstyle | 1m 52s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 49s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 17m 48s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 1m 16s | the patch passed | | :green_heart: | findbugs | 6m 2s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 0m 34s | hbase-hadoop-compat in the patch passed. | | :green_heart: | unit | 0m 40s | hbase-hadoop2-compat in the patch passed. | | :broken_heart: | unit | 328m 8s | hbase-server in the patch failed. | | :green_heart: | asflicense | 1m 26s | The patch does not generate ASF License warnings. | | | | 399m 24s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | | | hadoop.hbase.master.procedure.TestSCPWithReplicas | | | hadoop.hbase.master.TestMasterShutdown | | | hadoop.hbase.tool.TestSecureBulkLoadHFiles | | | hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory | | | hadoop.hbase.master.procedure.TestTruncateTableProcedure | | | hadoop.hbase.replication.TestReplicationSmallTestsSync | | | hadoop.hbase.tool.TestBulkLoadHFiles | | | hadoop.hbase.replication.TestReplicationKillSlaveRS | | | hadoop.hbase.master.procedure.TestProcedurePriority | | | hadoop.hbase.replication.TestReplicationSmallTests | | | hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure | | | hadoop.hbase.client.TestCloneSnapshotFromClientNormal | | | hadoop.hbase.client.TestFromClientSideWithCoprocessor | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-615/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/615 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux c976336c559f 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-615/out/precommit/personality/provided.sh | | git revision | master / 1655360951 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-615/4/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-615/4/testReport/ | | Max.
[jira] [Commented] (HBASE-22700) Incorrect timeout in recommended ZooKeeper configuration
[
https://issues.apache.org/jira/browse/HBASE-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933634#comment-16933634
]
Hudson commented on HBASE-22700:
Results for branch branch-2.2
[build #622 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/622/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/622//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/622//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/622//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/622//console].
> Incorrect timeout in recommended ZooKeeper configuration
>
>
> Key: HBASE-22700
> URL: https://issues.apache.org/jira/browse/HBASE-22700
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Peter Somogyi
>Assignee: maoling
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
>
> The [Recommended
> configuration|https://hbase.apache.org/book.html#recommended_configurations.zk]
> section for ZooKeeper states that the default zookeeper.session.timeout is 3
> minutes, however, the [default
> configuration|https://github.com/apache/hbase/blob/master/hbase-common/src/main/resources/hbase-default.xml#L372-L373]
> is 90 seconds(
>
> {code:java}
> /** Default value for ZooKeeper session timeout */
> public static final int DEFAULT_ZK_SESSION_TIMEOUT = 90 * 1000;
> ).
> {code}
>
> This section in the documentation should be modified to reflect the default
> configuration.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23039) HBCK2 bypass -r command does not work
[
https://issues.apache.org/jira/browse/HBASE-23039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933633#comment-16933633
]
Peter Somogyi commented on HBASE-23039:
---
{quote}Only effect bypass cmd?
{quote}
Yes, only bypass command.
> HBCK2 bypass -r command does not work
> -
>
> Key: HBASE-23039
> URL: https://issues.apache.org/jira/browse/HBASE-23039
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.0.0
>Reporter: Yi Mei
>Assignee: Yi Mei
>Priority: Major
> Fix For: hbase-operator-tools-2.0.0
>
> Attachments: HBASE-23039.001.patch
>
>
>
> The recursiveFlag is wrong:
> {code:java}
> boolean overrideFlag = commandLine.hasOption(override.getOpt());
> boolean recursiveFlag = commandLine.hasOption(override.getOpt());
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #631: HBASE-23035 Retain region to the last RegionServer make the failover …
Apache-HBase commented on issue #631: HBASE-23035 Retain region to the last RegionServer make the failover … URL: https://github.com/apache/hbase/pull/631#issuecomment-533234659 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 39s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :yellow_heart: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | :green_heart: | mvninstall | 5m 29s | master passed | | :green_heart: | compile | 0m 57s | master passed | | :green_heart: | checkstyle | 1m 16s | master passed | | :green_heart: | shadedjars | 4m 52s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 38s | master passed | | :blue_heart: | spotbugs | 4m 38s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 4m 35s | master passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 6m 5s | the patch passed | | :green_heart: | compile | 1m 11s | the patch passed | | :green_heart: | javac | 1m 11s | the patch passed | | :green_heart: | checkstyle | 1m 28s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 5m 39s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 17m 51s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 38s | the patch passed | | :green_heart: | findbugs | 4m 14s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 160m 11s | hbase-server in the patch passed. | | :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 222m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-631/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/631 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 05f79c16e557 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-631/out/precommit/personality/provided.sh | | git revision | master / db0049558e | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-631/4/testReport/ | | Max. process+thread count | 4602 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-631/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23051) Remove unneeded Mockito.mock invocations
[
https://issues.apache.org/jira/browse/HBASE-23051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi updated HBASE-23051:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to branch-2.1+.
Thanks for the reviews.
> Remove unneeded Mockito.mock invocations
>
>
> Key: HBASE-23051
> URL: https://issues.apache.org/jira/browse/HBASE-23051
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> ErrorProne fails the build using the new Mockito version.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile
> (default-testCompile) on project hbase-server: Compilation failure:
> Compilation failure:
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/MockMasterServices.java:[147,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR]
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedureControllers.java:[331,16]
> error: [CheckReturnValue] Ignored return value of method that is annotated
> with @CheckReturnValue
> [ERROR] (see https://errorprone.info/bugpattern/CheckReturnValue)
> [ERROR] Did you mean to remove this line?
> [ERROR] -> [Help 1] {noformat}
> [https://builds.apache.org/job/HBase%20Nightly/job/master/1456/artifact/output-jdk8-hadoop2/patch-compile-root.txt]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] petersomogyi merged pull request #641: HBASE-23051 Remove unneeded Mockito.mock invocations
petersomogyi merged pull request #641: HBASE-23051 Remove unneeded Mockito.mock invocations URL: https://github.com/apache/hbase/pull/641 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #595: HBASE-22971 Deprecated RSGroupAdminEndpoint and make RSGroup feature …
Apache-HBase commented on issue #595: HBASE-22971 Deprecated RSGroupAdminEndpoint and make RSGroup feature … URL: https://github.com/apache/hbase/pull/595#issuecomment-533230807 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 2m 9s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 11 new or modified test files. | ||| _ HBASE-22514 Compile Tests _ | | :green_heart: | mvninstall | 7m 58s | HBASE-22514 passed | | :green_heart: | compile | 1m 8s | HBASE-22514 passed | | :green_heart: | checkstyle | 1m 44s | HBASE-22514 passed | | :green_heart: | shadedjars | 5m 46s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 45s | HBASE-22514 passed | | :blue_heart: | spotbugs | 4m 50s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 4m 48s | HBASE-22514 passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 5m 25s | the patch passed | | :green_heart: | compile | 0m 58s | the patch passed | | :green_heart: | javac | 0m 58s | the patch passed | | :broken_heart: | checkstyle | 1m 25s | hbase-server: The patch generated 2 new + 202 unchanged - 4 fixed = 204 total (was 206) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 39s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 15m 52s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 37s | the patch passed | | :green_heart: | findbugs | 4m 45s | the patch passed | ||| _ Other Tests _ | | :broken_heart: | unit | 342m 55s | hbase-server in the patch failed. | | :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 407m 58s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | | | hadoop.hbase.master.TestMasterMetrics | | | hadoop.hbase.security.access.TestTablePermissions | | | hadoop.hbase.security.access.TestSnapshotScannerHDFSAclController | | | hadoop.hbase.client.TestSnapshotTemporaryDirectory | | | hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory | | | hadoop.hbase.replication.TestReplicationSmallTestsSync | | | hadoop.hbase.regionserver.TestRegionServerReportForDuty | | | hadoop.hbase.replication.TestReplicationSmallTests | | | hadoop.hbase.regionserver.TestRegionOpen | | | hadoop.hbase.replication.TestReplicationEndpoint | | | hadoop.hbase.namespace.TestNamespaceAuditor | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-595/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/595 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 424acc6094ca 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-595/out/precommit/personality/provided.sh | | git revision | HBASE-22514 / b91ef7c9dd | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-595/7/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-595/7/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-595/7/testReport/ | | Max. process+thread count | 5163 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-595/7/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message,
[GitHub] [hbase] virajjasani commented on a change in pull request #600: HBASE-22460 : Reopen regions with very high Store Ref Counts
virajjasani commented on a change in pull request #600: HBASE-22460 : Reopen
regions with very high Store Ref Counts
URL: https://github.com/apache/hbase/pull/600#discussion_r326124979
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionsRecoveryChore.java
##
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.master;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.ClusterMetrics;
+import org.apache.hadoop.hbase.RegionMetrics;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.ServerMetrics;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.Stoppable;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.PerClientRandomNonceGenerator;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.org.apache.commons.collections4.MapUtils;
+
+/**
+ * This chore, every time it runs, will try to recover regions with high store
ref count
+ * by reopening them
+ */
+@InterfaceAudience.Private
+public class RegionsRecoveryChore extends ScheduledChore {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(RegionsRecoveryChore.class);
+
+ private static final String REGIONS_RECOVERY_CHORE_NAME =
"RegionsRecoveryChore";
+
+ private static final String REGIONS_RECOVERY_INTERVAL =
+"hbase.master.regions.recovery.interval";
+ private static final String STORE_REF_COUNT_THRESHOLD =
"hbase.regions.recovery.store.count";
+
+ private static final int DEFAULT_REGIONS_RECOVERY_INTERVAL = 1200 * 1000; //
Default 20 min ?
+ private static final int DEFAULT_STORE_REF_COUNT_THRESHOLD = 256;
+
+ private static final String ERROR_REOPEN_REIONS_MSG =
+"Error reopening regions with high storeRefCount. ";
+
+ private final HMaster hMaster;
+ private final int storeRefCountThreshold;
+
+ private static final PerClientRandomNonceGenerator NONCE_GENERATOR =
+PerClientRandomNonceGenerator.get();
+
+ /**
+ * Construct RegionsRecoveryChore with provided params
+ *
+ * @param stopper When {@link Stoppable#isStopped()} is true, this chore
will cancel and cleanup
+ * @param configuration The configuration params to be used
+ * @param hMaster HMaster instance to initiate RegionTableRegions
+ */
+ RegionsRecoveryChore(final Stoppable stopper, final Configuration
configuration,
+ final HMaster hMaster) {
+
+super(REGIONS_RECOVERY_CHORE_NAME, stopper,
configuration.getInt(REGIONS_RECOVERY_INTERVAL,
+ DEFAULT_REGIONS_RECOVERY_INTERVAL));
+this.hMaster = hMaster;
+this.storeRefCountThreshold =
configuration.getInt(STORE_REF_COUNT_THRESHOLD,
+ DEFAULT_STORE_REF_COUNT_THRESHOLD);
+
+ }
+
+ @Override
+ protected void chore() {
+if (LOG.isTraceEnabled()) {
+ LOG.trace("Starting up Regions Recovery by reopening regions based on
storeRefCount...");
+}
+try {
+ final ClusterMetrics clusterMetrics = hMaster.getClusterMetrics();
+ final Map serverMetricsMap =
+clusterMetrics.getLiveServerMetrics();
+ final Map> tableToReopenRegionsMap =
+getTableToRegionsByRefCount(serverMetricsMap);
+ if (MapUtils.isNotEmpty(tableToReopenRegionsMap)) {
+tableToReopenRegionsMap.forEach((tableName, regionNames) -> {
+ try {
+LOG.warn("Reopening regions due to high refCount. TableName: {} ,
noOfRegions: {}",
+ tableName, regionNames.size());
+hMaster.reopenRegions(tableName, regionNames,
NONCE_GENERATOR.getNonceGroup(),
+ NONCE_GENERATOR.newNonce());
+ } catch (IOException e) {
+LOG.error("{} tableName: {}, regionNames: {}",
ERROR_REOPEN_REIONS_MSG,
+ tableName, regionNames, e);
+ }
+});
+ }
+} catch (Exception e) {
+ LOG.error("Error while reopening regions based on
[GitHub] [hbase] Apache-HBase commented on issue #641: HBASE-23051 Remove unneeded Mockito.mock invocations
Apache-HBase commented on issue #641: HBASE-23051 Remove unneeded Mockito.mock invocations URL: https://github.com/apache/hbase/pull/641#issuecomment-533199826 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | :green_heart: | mvninstall | 5m 58s | master passed | | :green_heart: | compile | 0m 59s | master passed | | :green_heart: | checkstyle | 1m 32s | master passed | | :green_heart: | shadedjars | 5m 2s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 39s | master passed | | :blue_heart: | spotbugs | 4m 35s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 4m 33s | master passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 5m 29s | the patch passed | | :green_heart: | compile | 0m 57s | the patch passed | | :green_heart: | javac | 0m 57s | the patch passed | | :green_heart: | checkstyle | 1m 27s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 5m 3s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 17m 41s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 35s | the patch passed | | :green_heart: | findbugs | 4m 47s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 157m 51s | hbase-server in the patch passed. | | :green_heart: | asflicense | 0m 25s | The patch does not generate ASF License warnings. | | | | 220m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-641/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/641 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 0dcf375b2661 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-641/out/precommit/personality/provided.sh | | git revision | master / 1655360951 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-641/2/testReport/ | | Max. process+thread count | 4490 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-641/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
