[GitHub] [hbase] liuml07 commented on a change in pull request #1485: HBASE-23969 Meta browser should show all `info` columns
liuml07 commented on a change in pull request #1485: URL: https://github.com/apache/hbase/pull/1485#discussion_r420579998 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/webapp/RegionReplicaInfo.java ## @@ -17,14 +17,22 @@ */ package org.apache.hadoop.hbase.master.webapp; +import static org.apache.hbase.thirdparty.org.apache.commons.collections4.ListUtils.emptyIfNull; Review comment: Thanks! I have fixed this. p.s. For a PR to Hadoop, I usually check the "checkstyle" report by Yetus and update the patch accordingly. I see the Yetus for HBase QA does not report "checkstyle" warnings in the comment table. I found there are report links in the "Console output" though. I'll follow there next time. :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] liuml07 commented on pull request #1485: HBASE-23969 Meta browser should show all `info` columns
liuml07 commented on pull request #1485: URL: https://github.com/apache/hbase/pull/1485#issuecomment-624475472 Thanks @ndimiduk I have pushed a new commit to address the comments. I tested locally again and it (still) seems good. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-24304) Separate a hbase-asyncfs module
[ https://issues.apache.org/jira/browse/HBASE-24304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-24304. --- Hadoop Flags: Reviewed Release Note: Added a new hbase-asyncfs module to hold the asynchronous dfs output stream implementation for implementing WAL. Resolution: Fixed > Separate a hbase-asyncfs module > --- > > Key: HBASE-24304 > URL: https://issues.apache.org/jira/browse/HBASE-24304 > Project: HBase > Issue Type: Improvement > Components: build, pom >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0 > > > To hold the async fs related class for WAL implementation as the hbase-server > module is too big. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24333) Backport HBASE-24304 "Separate a hbase-asyncfs module" to branch-2.x
Duo Zhang created HBASE-24333: - Summary: Backport HBASE-24304 "Separate a hbase-asyncfs module" to branch-2.x Key: HBASE-24333 URL: https://issues.apache.org/jira/browse/HBASE-24333 Project: HBase Issue Type: Sub-task Components: build, pom Reporter: Duo Zhang Assignee: Duo Zhang Fix For: 2.3.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] nyl3532016 commented on a change in pull request #1629: HBASE-24328 skip duplicate GCMultipleMergedRegionsProcedure while previous finished
nyl3532016 commented on a change in pull request #1629:
URL: https://github.com/apache/hbase/pull/1629#discussion_r420571160
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/GCMultipleMergedRegionsProcedure.java
##
@@ -78,6 +78,13 @@ protected Flow executeFromState(MasterProcedureEnv env,
GCMergedRegionsState sta
switch (state) {
case GC_MERGED_REGIONS_PREPARE:
// Nothing to do to prepare.
Review comment:
ok,thanks, I know. I will add this comment.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] liuml07 commented on a change in pull request #1485: HBASE-23969 Meta browser should show all `info` columns
liuml07 commented on a change in pull request #1485:
URL: https://github.com/apache/hbase/pull/1485#discussion_r420569058
##
File path: hbase-server/src/main/resources/hbase-webapps/master/table.jsp
##
@@ -387,68 +388,93 @@ if (fqtn != null && master.isInitialized()) {
}
}
%>
-
-
-RegionName
-Start Key
-End Key
-Replica ID
-RegionState
-ServerName
-
-<%
- final boolean metaScanHasMore;
- byte[] lastRow = null;
- try (final MetaBrowser.Results results = metaBrowser.getResults()) {
-for (final RegionReplicaInfo regionReplicaInfo : results) {
- lastRow = Optional.ofNullable(regionReplicaInfo)
-.map(RegionReplicaInfo::getRow)
-.orElse(null);
- if (regionReplicaInfo == null) {
-%>
-
-Null result
-
-<%
- continue;
-}
+
+
+
+ RegionName
+ Start Key
+ End Key
+ Replica ID
+ RegionState
+ Server
Review comment:
I like the idea. I have changed that. To make this table header fit in
one line, I have defined some column name variables, which can be reused.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun opened a new pull request #1663: Backport: HBASE-24316 GCMulitpleMergedRegionsProcedure is not idempotent (#1660)
huaxiangsun opened a new pull request #1663: URL: https://github.com/apache/hbase/pull/1663 It addresses couple issues: 1. Make sure deleteMergeQualifiers() does not delete the row if there is no columns with "merge" keyword. 2. GCMulitpleMergedRegionsProcedure now acquire an exclusive lock on the child region. Signed-off-by: stack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #1629: HBASE-24328 skip duplicate GCMultipleMergedRegionsProcedure while previous finished
huaxiangsun commented on a change in pull request #1629:
URL: https://github.com/apache/hbase/pull/1629#discussion_r420563488
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/GCMultipleMergedRegionsProcedure.java
##
@@ -78,6 +78,13 @@ protected Flow executeFromState(MasterProcedureEnv env,
GCMergedRegionsState sta
switch (state) {
case GC_MERGED_REGIONS_PREPARE:
// Nothing to do to prepare.
Review comment:
Hi @nyl3532016, can you put your above comment "If
GCMultipleMergedRegionsProcedure processing is slower than the CatalogJanitor's
scan interval, it will end resubmitting GCMultipleMergedRegionsProcedure for
the same region, we can skip duplicate GCMultipleMergedRegionsProcedure" to the
code above? It will help other developers's code reading easier. After that,
you can merge, thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] gjacoby126 commented on a change in pull request #1655: HBASE-24321 - Add writable MinVersions and read-only Scan to coproc S…
gjacoby126 commented on a change in pull request #1655:
URL: https://github.com/apache/hbase/pull/1655#discussion_r420561668
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanOptions.java
##
@@ -64,4 +66,13 @@ default void readAllVersions() {
void setKeepDeletedCells(KeepDeletedCells keepDeletedCells);
KeepDeletedCells getKeepDeletedCells();
+
+ int getMinVersions();
+
+ void setMinVersions(int minVersions);
+
+ /**
+ * Returns a read-only copy of the Scan object
Review comment:
Good point. Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] liuml07 commented on a change in pull request #1485: HBASE-23969 Meta browser should show all `info` columns
liuml07 commented on a change in pull request #1485:
URL: https://github.com/apache/hbase/pull/1485#discussion_r420561047
##
File path: hbase-server/src/main/resources/hbase-webapps/static/css/hbase.css
##
@@ -54,3 +54,7 @@ table.tablesorter thead tr .headerSortUp {
table.tablesorter thead tr .headerSortDown {
background-image: url(desc.gif);
}
+
+table.nowrap th, td {
Review comment:
Oh, right! No I was going to add the "nowrap" class only and changing
default td style may make other places unhappy.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1655: HBASE-24321 - Add writable MinVersions and read-only Scan to coproc S…
Apache-HBase commented on pull request #1655: URL: https://github.com/apache/hbase/pull/1655#issuecomment-624457085 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 2s | master passed | | +1 :green_heart: | compile | 1m 2s | master passed | | +1 :green_heart: | shadedjars | 5m 12s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 40s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 51s | the patch passed | | +1 :green_heart: | compile | 1m 1s | the patch passed | | +1 :green_heart: | javac | 1m 1s | the patch passed | | +1 :green_heart: | shadedjars | 5m 8s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 125m 21s | hbase-server in the patch passed. | | | | 149m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1655/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1655 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b864d7a2252a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3d96007c37 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1655/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1655/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1655/3/testReport/ | | Max. process+thread count | 4052 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1655/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1628: HBASE-24304 Separate a hbase-asyncfs module
Apache-HBase commented on pull request #1628: URL: https://github.com/apache/hbase/pull/1628#issuecomment-624456292 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 23s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 43s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 45s | master passed | | +1 :green_heart: | compile | 2m 59s | master passed | | +1 :green_heart: | shadedjars | 5m 50s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 19s | root in master failed. | | -0 :warning: | javadoc | 0m 17s | hbase-examples in master failed. | | -0 :warning: | javadoc | 0m 19s | hbase-mapreduce in master failed. | | -0 :warning: | javadoc | 0m 19s | hbase-rest in master failed. | | -0 :warning: | javadoc | 0m 40s | hbase-server in master failed. | | -0 :warning: | javadoc | 0m 58s | hbase-thrift in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 28s | the patch passed | | +1 :green_heart: | compile | 3m 2s | the patch passed | | +1 :green_heart: | javac | 3m 2s | the patch passed | | +1 :green_heart: | shadedjars | 5m 56s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 15s | hbase-asyncfs in the patch failed. | | -0 :warning: | javadoc | 0m 16s | root in the patch failed. | | -0 :warning: | javadoc | 0m 16s | hbase-examples in the patch failed. | | -0 :warning: | javadoc | 0m 19s | hbase-mapreduce in the patch failed. | | -0 :warning: | javadoc | 0m 18s | hbase-rest in the patch failed. | | -0 :warning: | javadoc | 0m 38s | hbase-server in the patch failed. | | -0 :warning: | javadoc | 0m 57s | hbase-thrift in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 235m 10s | root in the patch passed. | | | | 276m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1628 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 655b4d659802 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3d96007c37 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-examples.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-mapreduce.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-rest.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-asyncfs.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-examples.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-mapreduce.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-rest.txt | | javadoc | https://builds.apache.org/job/HBase-PreCom
[GitHub] [hbase] Apache-HBase commented on pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
Apache-HBase commented on pull request #1643: URL: https://github.com/apache/hbase/pull/1643#issuecomment-624452897 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 35s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 2m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1643/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1643 | | Optional Tests | | | uname | Linux 5543e1e5eca4 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 07077a3950 | | Max. process+thread count | 41 (vs. ulimit of 12500) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1643/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
Apache-HBase commented on pull request #1643: URL: https://github.com/apache/hbase/pull/1643#issuecomment-624452673 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] mattf-apache commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
mattf-apache commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420554402
##
File path: dev-support/create-release/release-build.sh
##
@@ -20,114 +20,180 @@
trap cleanup EXIT
# Source in utils.
-SELF=$(cd $(dirname $0) && pwd)
+SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=SCRIPTDIR/release-util.sh
. "$SELF/release-util.sh"
# Print usage and exit.
function exit_with_usage {
- cat << EOF
-Usage: release-build.sh
-Creates build deliverables from a tag/commit.
-Arguments:
- build Create binary packages and commit to
dist.apache.org/repos/dist/dev/hbase/
- publish-snapshot Publish snapshot release to Apache snapshots
- publish-release Publish a release to Apache release repo
-
-All other inputs are environment variables:
- GIT_REF - Release tag or commit to build from
- PACKAGE_VERSION - Release identifier in top level package directory (e.g.
2.1.2RC1)
- VERSION - (optional) Version of project being built (e.g. 2.1.2)
- ASF_USERNAME - Username of ASF committer account
- ASF_PASSWORD - Password of ASF committer account
- GPG_KEY - GPG key used to sign release artifacts
- GPG_PASSPHRASE - Passphrase for GPG key
- PROJECT - The project to build. No default.
-
-Set REPO environment to full path to repo to use
-to avoid re-downloading dependencies on each run.
+ cat <<'EOF'
+Usage: release-build.sh
+Creates release deliverables from a tag or commit.
+Argument: one of 'tag', 'publish-dist', 'publish-snapshot', or
'publish-release'
+ tag Prepares for release on specified git branch: Set release
version,
+update CHANGES and RELEASENOTES, create release tag,
+increment version for ongoing dev, and publish to Apache
git repo.
+ publish-dist Build and publish distribution packages (tarballs) to
Apache dist repo
+ publish-snapshot Build and publish maven artifacts snapshot release to
Apache snapshots repo
+ publish-release Build and publish maven artifacts release to Apache
release repo, and
+construct vote email from template
+
+All other inputs are environment variables. Please use do-release-docker.sh or
+do-release.sh to set up the needed environment variables. This script,
release-build.sh,
+is not intended to be called stand-alone, and such use is untested. The env
variables used are:
Review comment:
Getting late, will tackle this in the morning.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] mattf-apache commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
mattf-apache commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420553720
##
File path: dev-support/create-release/do-release.sh
##
@@ -66,26 +82,34 @@ function should_build {
fi
}
-if should_build "tag" && [ $SKIP_TAG = 0 ]; then
+if should_build "tag" && [ "$SKIP_TAG" = 0 ]; then
run_silent "Creating release tag $RELEASE_TAG..." "tag.log" \
-"$SELF/release-tag.sh"
- echo "It may take some time for the tag to be synchronized to github."
- echo "Press enter when you've verified that the new tag ($RELEASE_TAG) is
available."
- read
+"$SELF/release-build.sh" tag
+ if is_dry_run; then
+export TAG_SAME_DRY_RUN="true";
+ else
+echo "It may take some time for the tag to be synchronized to github."
+echo "Press enter when you've verified that the new tag ($RELEASE_TAG) is
available."
+read -r
+ fi
Review comment:
Done. See commit db4bab6f57d6 just added.
It waits in 30-second increments up to 5 minutes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] mattf-apache commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
mattf-apache commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420553720
##
File path: dev-support/create-release/do-release.sh
##
@@ -66,26 +82,34 @@ function should_build {
fi
}
-if should_build "tag" && [ $SKIP_TAG = 0 ]; then
+if should_build "tag" && [ "$SKIP_TAG" = 0 ]; then
run_silent "Creating release tag $RELEASE_TAG..." "tag.log" \
-"$SELF/release-tag.sh"
- echo "It may take some time for the tag to be synchronized to github."
- echo "Press enter when you've verified that the new tag ($RELEASE_TAG) is
available."
- read
+"$SELF/release-build.sh" tag
+ if is_dry_run; then
+export TAG_SAME_DRY_RUN="true";
+ else
+echo "It may take some time for the tag to be synchronized to github."
+echo "Press enter when you've verified that the new tag ($RELEASE_TAG) is
available."
+read -r
+ fi
Review comment:
Done. See commit db4bab6f57d6 just added.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
saintstack commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420552955
##
File path: dev-support/create-release/do-release.sh
##
@@ -66,26 +82,34 @@ function should_build {
fi
}
-if should_build "tag" && [ $SKIP_TAG = 0 ]; then
+if should_build "tag" && [ "$SKIP_TAG" = 0 ]; then
run_silent "Creating release tag $RELEASE_TAG..." "tag.log" \
-"$SELF/release-tag.sh"
- echo "It may take some time for the tag to be synchronized to github."
- echo "Press enter when you've verified that the new tag ($RELEASE_TAG) is
available."
- read
+"$SELF/release-build.sh" tag
+ if is_dry_run; then
+export TAG_SAME_DRY_RUN="true";
+ else
+echo "It may take some time for the tag to be synchronized to github."
+echo "Press enter when you've verified that the new tag ($RELEASE_TAG) is
available."
+read -r
+ fi
Review comment:
No hurry. Nice-to-have. Can come later too.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
saintstack commented on a change in pull request #1643: URL: https://github.com/apache/hbase/pull/1643#discussion_r420552985 ## File path: dev-support/create-release/do-release-docker.sh ## @@ -64,24 +65,26 @@ This script runs the release scripts inside a docker image. Options: -d [path]required. working directory. output will be written to "output" in here. - -n dry run mode. Checks and local builds, but does not upload anything. + -f "force" -- actually publish this release. Unless you specify '-f', it will + default to dry run mode, which checks and does local builds, but does not upload anything. Review comment: Good This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
saintstack commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420552836
##
File path: dev-support/create-release/do-release.sh
##
@@ -66,26 +82,34 @@ function should_build {
fi
}
-if should_build "tag" && [ $SKIP_TAG = 0 ]; then
+if should_build "tag" && [ "$SKIP_TAG" = 0 ]; then
run_silent "Creating release tag $RELEASE_TAG..." "tag.log" \
-"$SELF/release-tag.sh"
- echo "It may take some time for the tag to be synchronized to github."
- echo "Press enter when you've verified that the new tag ($RELEASE_TAG) is
available."
- read
+"$SELF/release-build.sh" tag
+ if is_dry_run; then
+export TAG_SAME_DRY_RUN="true";
+ else
+echo "It may take some time for the tag to be synchronized to github."
+echo "Press enter when you've verified that the new tag ($RELEASE_TAG) is
available."
+read -r
+ fi
else
echo "Skipping tag creation for $RELEASE_TAG."
fi
-if should_build "build"; then
- run_silent "Building ${PROJECT}..." "build.log" \
-"$SELF/release-build.sh" build
+if should_build "publish-dist"; then
+ run_silent "Publishing distribution packages (tarballs)" "publish-dist.log" \
+"$SELF/release-build.sh" publish-dist
else
- echo "Skipping build step."
+ echo "Skipping publish-dist step."
Review comment:
Good.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
saintstack commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420552786
##
File path: dev-support/create-release/release-build.sh
##
@@ -20,114 +20,180 @@
trap cleanup EXIT
# Source in utils.
-SELF=$(cd $(dirname $0) && pwd)
+SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=SCRIPTDIR/release-util.sh
. "$SELF/release-util.sh"
# Print usage and exit.
function exit_with_usage {
- cat << EOF
-Usage: release-build.sh
-Creates build deliverables from a tag/commit.
-Arguments:
- build Create binary packages and commit to
dist.apache.org/repos/dist/dev/hbase/
- publish-snapshot Publish snapshot release to Apache snapshots
- publish-release Publish a release to Apache release repo
-
-All other inputs are environment variables:
- GIT_REF - Release tag or commit to build from
- PACKAGE_VERSION - Release identifier in top level package directory (e.g.
2.1.2RC1)
- VERSION - (optional) Version of project being built (e.g. 2.1.2)
- ASF_USERNAME - Username of ASF committer account
- ASF_PASSWORD - Password of ASF committer account
- GPG_KEY - GPG key used to sign release artifacts
- GPG_PASSPHRASE - Passphrase for GPG key
- PROJECT - The project to build. No default.
-
-Set REPO environment to full path to repo to use
-to avoid re-downloading dependencies on each run.
+ cat <<'EOF'
+Usage: release-build.sh
+Creates release deliverables from a tag or commit.
+Argument: one of 'tag', 'publish-dist', 'publish-snapshot', or
'publish-release'
+ tag Prepares for release on specified git branch: Set release
version,
+update CHANGES and RELEASENOTES, create release tag,
+increment version for ongoing dev, and publish to Apache
git repo.
+ publish-dist Build and publish distribution packages (tarballs) to
Apache dist repo
+ publish-snapshot Build and publish maven artifacts snapshot release to
Apache snapshots repo
+ publish-release Build and publish maven artifacts release to Apache
release repo, and
Review comment:
k. This sounds like it works as it used to which sort of basically
worked.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
saintstack commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420552406
##
File path: dev-support/create-release/release-build.sh
##
@@ -20,114 +20,180 @@
trap cleanup EXIT
# Source in utils.
-SELF=$(cd $(dirname $0) && pwd)
+SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=SCRIPTDIR/release-util.sh
. "$SELF/release-util.sh"
# Print usage and exit.
function exit_with_usage {
- cat << EOF
-Usage: release-build.sh
-Creates build deliverables from a tag/commit.
-Arguments:
- build Create binary packages and commit to
dist.apache.org/repos/dist/dev/hbase/
- publish-snapshot Publish snapshot release to Apache snapshots
- publish-release Publish a release to Apache release repo
-
-All other inputs are environment variables:
- GIT_REF - Release tag or commit to build from
- PACKAGE_VERSION - Release identifier in top level package directory (e.g.
2.1.2RC1)
- VERSION - (optional) Version of project being built (e.g. 2.1.2)
- ASF_USERNAME - Username of ASF committer account
- ASF_PASSWORD - Password of ASF committer account
- GPG_KEY - GPG key used to sign release artifacts
- GPG_PASSPHRASE - Passphrase for GPG key
- PROJECT - The project to build. No default.
-
-Set REPO environment to full path to repo to use
-to avoid re-downloading dependencies on each run.
+ cat <<'EOF'
+Usage: release-build.sh
+Creates release deliverables from a tag or commit.
+Argument: one of 'tag', 'publish-dist', 'publish-snapshot', or
'publish-release'
+ tag Prepares for release on specified git branch: Set release
version,
+update CHANGES and RELEASENOTES, create release tag,
+increment version for ongoing dev, and publish to Apache
git repo.
+ publish-dist Build and publish distribution packages (tarballs) to
Apache dist repo
+ publish-snapshot Build and publish maven artifacts snapshot release to
Apache snapshots repo
+ publish-release Build and publish maven artifacts release to Apache
release repo, and
+construct vote email from template
+
+All other inputs are environment variables. Please use do-release-docker.sh or
+do-release.sh to set up the needed environment variables. This script,
release-build.sh,
+is not intended to be called stand-alone, and such use is untested. The env
variables used are:
Review comment:
Sweet
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
saintstack commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420552310
##
File path: dev-support/create-release/release-build.sh
##
@@ -20,114 +20,180 @@
trap cleanup EXIT
# Source in utils.
-SELF=$(cd $(dirname $0) && pwd)
+SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=SCRIPTDIR/release-util.sh
. "$SELF/release-util.sh"
# Print usage and exit.
function exit_with_usage {
- cat << EOF
-Usage: release-build.sh
-Creates build deliverables from a tag/commit.
-Arguments:
- build Create binary packages and commit to
dist.apache.org/repos/dist/dev/hbase/
- publish-snapshot Publish snapshot release to Apache snapshots
- publish-release Publish a release to Apache release repo
-
-All other inputs are environment variables:
- GIT_REF - Release tag or commit to build from
- PACKAGE_VERSION - Release identifier in top level package directory (e.g.
2.1.2RC1)
- VERSION - (optional) Version of project being built (e.g. 2.1.2)
- ASF_USERNAME - Username of ASF committer account
- ASF_PASSWORD - Password of ASF committer account
- GPG_KEY - GPG key used to sign release artifacts
- GPG_PASSPHRASE - Passphrase for GPG key
- PROJECT - The project to build. No default.
-
-Set REPO environment to full path to repo to use
-to avoid re-downloading dependencies on each run.
+ cat <<'EOF'
+Usage: release-build.sh
+Creates release deliverables from a tag or commit.
+Argument: one of 'tag', 'publish-dist', 'publish-snapshot', or
'publish-release'
+ tag Prepares for release on specified git branch: Set release
version,
+update CHANGES and RELEASENOTES, create release tag,
+increment version for ongoing dev, and publish to Apache
git repo.
+ publish-dist Build and publish distribution packages (tarballs) to
Apache dist repo
+ publish-snapshot Build and publish maven artifacts snapshot release to
Apache snapshots repo
+ publish-release Build and publish maven artifacts release to Apache
release repo, and
+construct vote email from template
+
+All other inputs are environment variables. Please use do-release-docker.sh or
+do-release.sh to set up the needed environment variables. This script,
release-build.sh,
+is not intended to be called stand-alone, and such use is untested. The env
variables used are:
+
+Used for 'tag' and 'publish' stages:
+ PROJECT - The project to build. No default.
+ RELEASE_VERSION - Version used in pom files for release (e.g. 2.1.2)
+Required for 'tag'; defaults for 'publish' to the version in pom at GIT_REF
+ RELEASE_TAG - Name of release tag (e.g. 2.1.2RC0), also used by
+publish-dist as package version name in dist directory path
+ ASF_USERNAME - Username of ASF committer account
+ ASF_PASSWORD - Password of ASF committer account
+ DRY_RUN - 1:true (default), 0:false. If "1", does almost all the work, but
doesn't actually
+publish anything to upstream source or object repositories. It defaults to
"1", so if you want
+to actually publish you have to set '-f' (force) flag in do-release.sh or
do-release-docker.sh.
+
+Used only for 'tag':
+ GIT_NAME - Name to use with git
+ GIT_EMAIL - E-mail address to use with git
+ GIT_BRANCH - Git branch on which to make release. Tag is always placed at
HEAD of this branch.
+ NEXT_VERSION - Development version after release (e.g. 2.1.3-SNAPSHOT)
+
+Used only for 'publish':
+ GIT_REF - Release tag or commit to build from (defaults to $RELEASE_TAG;
only need to
+separately define GIT_REF if RELEASE_TAG is not actually present as a tag
at publish time)
+If both RELEASE_TAG and GIT_REF are undefined it will default to HEAD of
master.
+ GPG_KEY - GPG key id (usually email addr) used to sign release artifacts
+ GPG_PASSPHRASE - Passphrase for GPG key
+ REPO - Set to full path of a directory to use as maven local repo
(dependencies cache)
+to avoid re-downloading dependencies for each stage. It is automatically
set if you
+request full sequence of stages (tag, publish-dist, publish-release) in
do-release.sh.
For example:
- $ PROJECT="hbase-operator-tools" ASF_USERNAME=NAME ASF_PASSWORD=PASSWORD
GPG_PASSPHRASE=PASSWORD GPG_KEY=st...@apache.org ./release-build.sh build
+ $ PROJECT="hbase-operator-tools" ASF_USERNAME=NAME ASF_PASSWORD=PASSWORD
GPG_PASSPHRASE=PASSWORD GPG_KEY=st...@apache.org ./release-build.sh publish-dist
EOF
exit 1
}
set -e
function cleanup {
- echo "Cleaning up temp settings file." >&2
- rm "${tmp_settings}" &> /dev/null || true
# If REPO was set, then leave things be. Otherwise if we defined a repo
clean it out.
- if [[ -z "${REPO}" ]] && [[ -n "${tmp_repo}" ]]; then
-echo "Cleaning up temp repo in '${tmp_repo}'. set REPO to reuse
downloads." >&2
-rm -rf "${tmp_repo}" &> /dev/null || true
+ if [[ -z "${REPO}" ]] && [[ -n "${MAVEN_LOCAL_REPO}" ]]; then
[GitHub] [hbase] saintstack commented on pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
saintstack commented on pull request #1643: URL: https://github.com/apache/hbase/pull/1643#issuecomment-624448967 @busbey and @ndimiduk You both are about to RM. You could review this as sub-task of RM'ing but may I suggest that you'll have your hands full RM'ing anyways; just take what is here and build on its benefit? It might be a bit hard to review because it does some renaming toward improved clarity and moves some actions around to make more sensible grouping. Just a suggestion. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #1628: HBASE-24304 Separate a hbase-asyncfs module
saintstack commented on pull request #1628: URL: https://github.com/apache/hbase/pull/1628#issuecomment-624446854 Approved for master. You think this good for branch-2 and branch-2.3 @Apache9 . I suppose it ok. Go for it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #1415: HBASE-24051 Allows indirect inheritance to CanUnbuffer
saintstack commented on pull request #1415: URL: https://github.com/apache/hbase/pull/1415#issuecomment-624446446 @shenshengli have you seen the @ndimiduk comment above? Any response? Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-23785) Update docs re: Hadoop compatibility of 2.3.x release
[ https://issues.apache.org/jira/browse/HBASE-23785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100336#comment-17100336 ] Michael Stack edited comment on HBASE-23785 at 5/6/20, 5:12 AM: Needs note on new zk version and upgrade issues if coming from old version? See HBASE-24132 for a recommendattion (can re-resolve HBASE-24132 when this is done?) was (Author: stack): Needs note on new zk version and upgrade issues if coming from old version? > Update docs re: Hadoop compatibility of 2.3.x release > - > > Key: HBASE-23785 > URL: https://issues.apache.org/jira/browse/HBASE-23785 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Nick Dimiduk >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0 > > > Update the [book|http://hbase.apache.org/book.html#hadoop] regarding our > Hadoop compatibility for the 2.3 release lines. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1657: HBASE-24331 [Flakey Test] TestJMXListener rmi port clash
Apache-HBase commented on pull request #1657: URL: https://github.com/apache/hbase/pull/1657#issuecomment-62946 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 38s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 7s | branch-2 passed | | +1 :green_heart: | spotbugs | 1m 57s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 10s | the patch passed | | +1 :green_heart: | checkstyle | 1m 3s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 39s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 32m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1657/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1657 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 6c7e3d1f8bae 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 99e96cca3b | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1657/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-23976) [flakey test] TestVerifyBucketCacheFile
[
https://issues.apache.org/jira/browse/HBASE-23976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-23976.
---
Fix Version/s: 2.3.0
3.0.0-alpha-1
Hadoop Flags: Reviewed
Assignee: Nick Dimiduk
Resolution: Fixed
Pushed to branch-2.3+. Lets see how it does. Thanks for review and testing
[~weichiu].
> [flakey test] TestVerifyBucketCacheFile
> ---
>
> Key: HBASE-23976
> URL: https://issues.apache.org/jira/browse/HBASE-23976
> Project: HBase
> Issue Type: Test
> Components: regionserver, test
>Affects Versions: 3.0.0-alpha-1
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
>
> I see sporadic failures in this test class. Sometimes a failure on
> {{assertTrue(file.delete())}}, an inconsistent annoyance. However, this one
> looks more sinister.
> {noformat}
> 2020-03-12 12:11:35,059 ERROR [Time-limited test] bucket.BucketCache(312):
> Can't restore from
> file[/Users/ndimiduk/repos/apache/hbase/hbase-server/target/test-data/5e5c5f5f-d5c2-94b2-8ce9-cf561f4f19f7/bucket.persistence]
> because of
> java.io.IOException: Mismatch of checksum! The persistent checksum is
> ���Bk���2�Ӏk, but the calculate checksum is
> �o���r��w��c��4
> at
> org.apache.hadoop.hbase.io.hfile.bucket.PersistentIOEngine.verifyFileIntegrity(PersistentIOEngine.java:55)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.parsePB(BucketCache.java:1158)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.retrieveFromFile(BucketCache.java:1106)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.(BucketCache.java:310)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.(BucketCache.java:258)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.TestVerifyBucketCacheFile.testRetrieveFromFile(TestVerifyBucketCacheFile.java:116)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] jojochuang commented on pull request #1342: HBASE-23976 Use nio for file manipulation in TestVerifyBucketCacheFile
jojochuang commented on pull request #1342: URL: https://github.com/apache/hbase/pull/1342#issuecomment-624440493 +1 although I don't understand why, but the test no longer fails. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1662: Backport: HBASE-24273 HBCK's "Orphan Regions on FileSystem" reports regions wit…
Apache-HBase commented on pull request #1662: URL: https://github.com/apache/hbase/pull/1662#issuecomment-624437948 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 19s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 50s | branch-2.3 passed | | +1 :green_heart: | compile | 1m 42s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 7m 52s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 59s | hbase-server in branch-2.3 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 35s | the patch passed | | +1 :green_heart: | compile | 1m 31s | the patch passed | | +1 :green_heart: | javac | 1m 31s | the patch passed | | +1 :green_heart: | shadedjars | 8m 1s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 58s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 221m 25s | hbase-server in the patch passed. | | | | 260m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1662/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1662 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 93d5ed425086 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / bee66bd22a | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1662/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1662/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1662/1/testReport/ | | Max. process+thread count | 2942 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1662/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24331) [Flakey Test] TestJMXListener rmi port clash
[ https://issues.apache.org/jira/browse/HBASE-24331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100459#comment-17100459 ] Michael Stack commented on HBASE-24331: --- [~markrmiller] I pushed the parent issue which has support for the PR in here. Moves the randomFreePort stuff back to hbase-common like you do but also adds BindException handling. Expand the bit in your patch where it finds other impls of randomFreePort and converts them to the 'good' impl? Lets get it in. > [Flakey Test] TestJMXListener rmi port clash > > > Key: HBASE-24331 > URL: https://issues.apache.org/jira/browse/HBASE-24331 > Project: HBase > Issue Type: Sub-task > Components: flakies, test >Reporter: Michael Stack >Priority: Major > > The TestJMXListener can fail because the random port it wants to put the jmx > listener on is occupied when it goes to run. Handle this case in test startup. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24307) [Flakey Tests] krb server for secure thrift tests throws BindException
[
https://issues.apache.org/jira/browse/HBASE-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-24307.
---
Assignee: Michael Stack
Resolution: Fixed
Pushed on branch-2.3+
> [Flakey Tests] krb server for secure thrift tests throws BindException
> --
>
> Key: HBASE-24307
> URL: https://issues.apache.org/jira/browse/HBASE-24307
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
>
> Thought I'd put all the BindExceptions in thrift to sleep but just came
> across this one putting up the *krb* server used by the secure thrift http
> tests. There are two tests that use krb:
> {code}
>
> ---
> Test set: org.apache.hadoop.hbase.thrift.TestThriftSpnegoHttpFallbackServer
>
> ---
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.042 s <<<
> FAILURE! - in
> org.apache.hadoop.hbase.thrift.TestThriftSpnegoHttpFallbackServer
> org.apache.hadoop.hbase.thrift.TestThriftSpnegoHttpFallbackServer Time
> elapsed: 0.017 s <<< ERROR!
> org.apache.kerby.kerberos.kerb.KrbException: Failed to start KDC-Server
>at
> org.apache.hadoop.hbase.thrift.TestThriftSpnegoHttpFallbackServer.setUpBeforeClass(TestThriftSpnegoHttpFallbackServer.java:137)
> Caused by: java.net.BindException: Address already in use (Bind failed)
>at
> org.apache.hadoop.hbase.thrift.TestThriftSpnegoHttpFallbackServer.setUpBeforeClass(TestThriftSpnegoHttpFallbackServer.java:137)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-24118) [Flakey Tests] TestCloseRegionWhileRSCrash
[
https://issues.apache.org/jira/browse/HBASE-24118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-24118.
---
Resolution: Fixed
Re-resolving.
For those trying to follow-behind, there were FOUR commits against this issue.
{code}
24c995da97 HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash Addendum; add
back bit missing off end of test.
0e30e00c56 HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash Reapply but
as an @Ignore for the flakey test.
0dc2712c90 Revert "HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash"
Reverting in favor of adding an @Ignore on this test until root cause of
flakyness HBASE-24117 is addressed.
9985c06647 HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash
{code}
The oldest was my remove of problematic bit of test. The second a revert of my
change after Duo pointed out I'd removed the important bit about the test and
suggested I restore and instead put an @Ignore on the test instead So
second commit if revert of first. The third is addition of the @Ignore. The
forth is me adding back the bit I should have restored somewhere in commit #2
or #3.
> [Flakey Tests] TestCloseRegionWhileRSCrash
> --
>
> Key: HBASE-24118
> URL: https://issues.apache.org/jira/browse/HBASE-24118
> Project: HBase
> Issue Type: Test
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
> Attachments:
> 0001-HBASE-24118-Flakey-Tests-TestCloseRegionWhileRSCrash.patch
>
>
> TestCloseRegionWhileRSCrash is flakey because of HBASE-24117 -- because
> moved Region may not online in the end. Remove the last bit of the test...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1593: HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry …
Apache-HBase commented on pull request #1593: URL: https://github.com/apache/hbase/pull/1593#issuecomment-624432537 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 36s | master passed | | +1 :green_heart: | compile | 1m 47s | master passed | | +1 :green_heart: | shadedjars | 5m 11s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 21s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 26s | the patch passed | | +1 :green_heart: | compile | 1m 42s | the patch passed | | +1 :green_heart: | javac | 1m 42s | the patch passed | | +1 :green_heart: | shadedjars | 4m 53s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 20s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 22s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 2s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 134m 43s | hbase-server in the patch passed. | | | | 164m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1593 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d6b79fa1ed48 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3d96007c37 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/6/testReport/ | | Max. process+thread count | 4390 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/6/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Reopened] (HBASE-24118) [Flakey Tests] TestCloseRegionWhileRSCrash
[
https://issues.apache.org/jira/browse/HBASE-24118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack reopened HBASE-24118:
---
Reopening. I needed to undo more; add back the problematic bit:
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestCloseRegionWhileRSCrash.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestCloseRegionWhileRSCrash.java
index 7eb453d271..4761991e75 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestCloseRegionWhileRSCrash.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestCloseRegionWhileRSCrash.java
@@ -202,5 +202,10 @@ public class TestCloseRegionWhileRSCrash {
try (Table table = UTIL.getConnection().getTable(TABLE_NAME)) {
table.put(new Put(Bytes.toBytes(1)).addColumn(CF, Bytes.toBytes("cq"),
Bytes.toBytes(1)));
}
+// Make sure that the region is online, it may not be on the original
target server, as we will
+// set forceNewPlan to true if there is a server crash.
+try (Table table = UTIL.getConnection().getTable(TABLE_NAME)) {
+ table.put(new Put(Bytes.toBytes(1)).addColumn(CF, Bytes.toBytes("cq"),
Bytes.toBytes(1)));
+}
}
}
> [Flakey Tests] TestCloseRegionWhileRSCrash
> --
>
> Key: HBASE-24118
> URL: https://issues.apache.org/jira/browse/HBASE-24118
> Project: HBase
> Issue Type: Test
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
> Attachments:
> 0001-HBASE-24118-Flakey-Tests-TestCloseRegionWhileRSCrash.patch
>
>
> TestCloseRegionWhileRSCrash is flakey because of HBASE-24117 -- because
> moved Region may not online in the end. Remove the last bit of the test...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HBASE-24118) [Flakey Tests] TestCloseRegionWhileRSCrash
[
https://issues.apache.org/jira/browse/HBASE-24118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100446#comment-17100446
]
Michael Stack edited comment on HBASE-24118 at 5/6/20, 4:12 AM:
Reopening. I needed to undo more; add back the problematic bit:
{code}
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestCloseRegionWhileRSCrash.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestCloseRegionWhileRSCrash.java
index 7eb453d271..4761991e75 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestCloseRegionWhileRSCrash.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestCloseRegionWhileRSCrash.java
@@ -202,5 +202,10 @@ public class TestCloseRegionWhileRSCrash {
try (Table table = UTIL.getConnection().getTable(TABLE_NAME)) {
table.put(new Put(Bytes.toBytes(1)).addColumn(CF, Bytes.toBytes("cq"),
Bytes.toBytes(1)));
}
+// Make sure that the region is online, it may not be on the original
target server, as we will
+// set forceNewPlan to true if there is a server crash.
+try (Table table = UTIL.getConnection().getTable(TABLE_NAME)) {
+ table.put(new Put(Bytes.toBytes(1)).addColumn(CF, Bytes.toBytes("cq"),
Bytes.toBytes(1)));
+}
}
}
{code}
was (Author: stack):
Reopening. I needed to undo more; add back the problematic bit:
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestCloseRegionWhileRSCrash.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestCloseRegionWhileRSCrash.java
index 7eb453d271..4761991e75 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestCloseRegionWhileRSCrash.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestCloseRegionWhileRSCrash.java
@@ -202,5 +202,10 @@ public class TestCloseRegionWhileRSCrash {
try (Table table = UTIL.getConnection().getTable(TABLE_NAME)) {
table.put(new Put(Bytes.toBytes(1)).addColumn(CF, Bytes.toBytes("cq"),
Bytes.toBytes(1)));
}
+// Make sure that the region is online, it may not be on the original
target server, as we will
+// set forceNewPlan to true if there is a server crash.
+try (Table table = UTIL.getConnection().getTable(TABLE_NAME)) {
+ table.put(new Put(Bytes.toBytes(1)).addColumn(CF, Bytes.toBytes("cq"),
Bytes.toBytes(1)));
+}
}
}
> [Flakey Tests] TestCloseRegionWhileRSCrash
> --
>
> Key: HBASE-24118
> URL: https://issues.apache.org/jira/browse/HBASE-24118
> Project: HBase
> Issue Type: Test
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
> Attachments:
> 0001-HBASE-24118-Flakey-Tests-TestCloseRegionWhileRSCrash.patch
>
>
> TestCloseRegionWhileRSCrash is flakey because of HBASE-24117 -- because
> moved Region may not online in the end. Remove the last bit of the test...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1655: HBASE-24321 - Add writable MinVersions and read-only Scan to coproc S…
Apache-HBase commented on pull request #1655: URL: https://github.com/apache/hbase/pull/1655#issuecomment-624431932 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 48s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 24s | master passed | | +1 :green_heart: | checkstyle | 1m 28s | master passed | | +1 :green_heart: | spotbugs | 2m 31s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 37s | the patch passed | | -0 :warning: | checkstyle | 1m 32s | hbase-server: The patch generated 2 new + 91 unchanged - 0 fixed = 93 total (was 91) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 14m 46s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 34s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 41m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1655/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1655 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 99aea058218b 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3d96007c37 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1655/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1655/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #1629: HBASE-24328 skip duplicate GCMultipleMergedRegionsProcedure while previous finished
huaxiangsun commented on a change in pull request #1629:
URL: https://github.com/apache/hbase/pull/1629#discussion_r419797107
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/GCMultipleMergedRegionsProcedure.java
##
@@ -78,6 +78,13 @@ protected Flow executeFromState(MasterProcedureEnv env,
GCMergedRegionsState sta
switch (state) {
case GC_MERGED_REGIONS_PREPARE:
// Nothing to do to prepare.
Review comment:
Can you review the comment line and new comment? Briefly explain what
the code does.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1593: HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry …
Apache-HBase commented on pull request #1593: URL: https://github.com/apache/hbase/pull/1593#issuecomment-624430676 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 53s | master passed | | +1 :green_heart: | compile | 1m 56s | master passed | | +1 :green_heart: | shadedjars | 5m 12s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 17s | hbase-common in master failed. | | -0 :warning: | javadoc | 0m 26s | hbase-client in master failed. | | -0 :warning: | javadoc | 0m 39s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 55s | the patch passed | | +1 :green_heart: | compile | 1m 55s | the patch passed | | +1 :green_heart: | javac | 1m 55s | the patch passed | | +1 :green_heart: | shadedjars | 5m 6s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 17s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 24s | hbase-client in the patch failed. | | -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 33s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 2s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 123m 54s | hbase-server in the patch passed. | | | | 155m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1593 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3705b7c31489 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3d96007c37 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/6/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/6/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/6/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/6/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/6/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/6/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/6/testReport/ | | Max. process+thread count | 4196 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/6/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1662: Backport: HBASE-24273 HBCK's "Orphan Regions on FileSystem" reports regions wit…
Apache-HBase commented on pull request #1662: URL: https://github.com/apache/hbase/pull/1662#issuecomment-624430786 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 52s | branch-2.3 passed | | +1 :green_heart: | compile | 0m 57s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 4m 50s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 53s | the patch passed | | +1 :green_heart: | compile | 1m 23s | the patch passed | | +1 :green_heart: | javac | 1m 23s | the patch passed | | +1 :green_heart: | shadedjars | 4m 57s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 201m 45s | hbase-server in the patch passed. | | | | 225m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1662/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1662 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d3d0b3734f08 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / bee66bd22a | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1662/1/testReport/ | | Max. process+thread count | 2734 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1662/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] mattf-apache commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
mattf-apache commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420526164
##
File path: dev-support/create-release/release-build.sh
##
@@ -20,114 +20,180 @@
trap cleanup EXIT
# Source in utils.
-SELF=$(cd $(dirname $0) && pwd)
+SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=SCRIPTDIR/release-util.sh
. "$SELF/release-util.sh"
# Print usage and exit.
function exit_with_usage {
- cat << EOF
-Usage: release-build.sh
-Creates build deliverables from a tag/commit.
-Arguments:
- build Create binary packages and commit to
dist.apache.org/repos/dist/dev/hbase/
- publish-snapshot Publish snapshot release to Apache snapshots
- publish-release Publish a release to Apache release repo
-
-All other inputs are environment variables:
- GIT_REF - Release tag or commit to build from
- PACKAGE_VERSION - Release identifier in top level package directory (e.g.
2.1.2RC1)
- VERSION - (optional) Version of project being built (e.g. 2.1.2)
- ASF_USERNAME - Username of ASF committer account
- ASF_PASSWORD - Password of ASF committer account
- GPG_KEY - GPG key used to sign release artifacts
- GPG_PASSPHRASE - Passphrase for GPG key
- PROJECT - The project to build. No default.
-
-Set REPO environment to full path to repo to use
-to avoid re-downloading dependencies on each run.
+ cat <<'EOF'
+Usage: release-build.sh
+Creates release deliverables from a tag or commit.
+Argument: one of 'tag', 'publish-dist', 'publish-snapshot', or
'publish-release'
+ tag Prepares for release on specified git branch: Set release
version,
+update CHANGES and RELEASENOTES, create release tag,
+increment version for ongoing dev, and publish to Apache
git repo.
+ publish-dist Build and publish distribution packages (tarballs) to
Apache dist repo
+ publish-snapshot Build and publish maven artifacts snapshot release to
Apache snapshots repo
+ publish-release Build and publish maven artifacts release to Apache
release repo, and
+construct vote email from template
+
+All other inputs are environment variables. Please use do-release-docker.sh or
+do-release.sh to set up the needed environment variables. This script,
release-build.sh,
+is not intended to be called stand-alone, and such use is untested. The env
variables used are:
+
+Used for 'tag' and 'publish' stages:
+ PROJECT - The project to build. No default.
+ RELEASE_VERSION - Version used in pom files for release (e.g. 2.1.2)
+Required for 'tag'; defaults for 'publish' to the version in pom at GIT_REF
+ RELEASE_TAG - Name of release tag (e.g. 2.1.2RC0), also used by
+publish-dist as package version name in dist directory path
+ ASF_USERNAME - Username of ASF committer account
+ ASF_PASSWORD - Password of ASF committer account
+ DRY_RUN - 1:true (default), 0:false. If "1", does almost all the work, but
doesn't actually
+publish anything to upstream source or object repositories. It defaults to
"1", so if you want
+to actually publish you have to set '-f' (force) flag in do-release.sh or
do-release-docker.sh.
+
+Used only for 'tag':
+ GIT_NAME - Name to use with git
+ GIT_EMAIL - E-mail address to use with git
+ GIT_BRANCH - Git branch on which to make release. Tag is always placed at
HEAD of this branch.
+ NEXT_VERSION - Development version after release (e.g. 2.1.3-SNAPSHOT)
+
+Used only for 'publish':
+ GIT_REF - Release tag or commit to build from (defaults to $RELEASE_TAG;
only need to
+separately define GIT_REF if RELEASE_TAG is not actually present as a tag
at publish time)
+If both RELEASE_TAG and GIT_REF are undefined it will default to HEAD of
master.
+ GPG_KEY - GPG key id (usually email addr) used to sign release artifacts
+ GPG_PASSPHRASE - Passphrase for GPG key
+ REPO - Set to full path of a directory to use as maven local repo
(dependencies cache)
+to avoid re-downloading dependencies for each stage. It is automatically
set if you
+request full sequence of stages (tag, publish-dist, publish-release) in
do-release.sh.
For example:
- $ PROJECT="hbase-operator-tools" ASF_USERNAME=NAME ASF_PASSWORD=PASSWORD
GPG_PASSPHRASE=PASSWORD GPG_KEY=st...@apache.org ./release-build.sh build
+ $ PROJECT="hbase-operator-tools" ASF_USERNAME=NAME ASF_PASSWORD=PASSWORD
GPG_PASSPHRASE=PASSWORD GPG_KEY=st...@apache.org ./release-build.sh publish-dist
EOF
exit 1
}
set -e
function cleanup {
- echo "Cleaning up temp settings file." >&2
- rm "${tmp_settings}" &> /dev/null || true
# If REPO was set, then leave things be. Otherwise if we defined a repo
clean it out.
- if [[ -z "${REPO}" ]] && [[ -n "${tmp_repo}" ]]; then
-echo "Cleaning up temp repo in '${tmp_repo}'. set REPO to reuse
downloads." >&2
-rm -rf "${tmp_repo}" &> /dev/null || true
+ if [[ -z "${REPO}" ]] && [[ -n "${MAVEN_LOCAL_REPO}" ]]; the
[GitHub] [hbase] mattf-apache commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
mattf-apache commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420508959
##
File path: dev-support/create-release/release-build.sh
##
@@ -20,114 +20,180 @@
trap cleanup EXIT
# Source in utils.
-SELF=$(cd $(dirname $0) && pwd)
+SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=SCRIPTDIR/release-util.sh
. "$SELF/release-util.sh"
# Print usage and exit.
function exit_with_usage {
- cat << EOF
-Usage: release-build.sh
-Creates build deliverables from a tag/commit.
-Arguments:
- build Create binary packages and commit to
dist.apache.org/repos/dist/dev/hbase/
- publish-snapshot Publish snapshot release to Apache snapshots
- publish-release Publish a release to Apache release repo
-
-All other inputs are environment variables:
- GIT_REF - Release tag or commit to build from
- PACKAGE_VERSION - Release identifier in top level package directory (e.g.
2.1.2RC1)
- VERSION - (optional) Version of project being built (e.g. 2.1.2)
- ASF_USERNAME - Username of ASF committer account
- ASF_PASSWORD - Password of ASF committer account
- GPG_KEY - GPG key used to sign release artifacts
- GPG_PASSPHRASE - Passphrase for GPG key
- PROJECT - The project to build. No default.
-
-Set REPO environment to full path to repo to use
-to avoid re-downloading dependencies on each run.
+ cat <<'EOF'
+Usage: release-build.sh
+Creates release deliverables from a tag or commit.
+Argument: one of 'tag', 'publish-dist', 'publish-snapshot', or
'publish-release'
+ tag Prepares for release on specified git branch: Set release
version,
+update CHANGES and RELEASENOTES, create release tag,
+increment version for ongoing dev, and publish to Apache
git repo.
+ publish-dist Build and publish distribution packages (tarballs) to
Apache dist repo
+ publish-snapshot Build and publish maven artifacts snapshot release to
Apache snapshots repo
+ publish-release Build and publish maven artifacts release to Apache
release repo, and
Review comment:
What one should do to run a release candidate, is invoke
`do-release-docker.sh` (preferably) or `do-release.sh` (if there's a reason not
to use docker). I agree with your comments here and in the next item below,
that this script must clearly be documented as NOT the entrance script. And
yet, the many env variables that get passed around as inter-script
communication do need to be documented somewhere. Let me see if I can move
them to `do-release.sh`, and make the comments and the README more useful.
On a completely unrelated note, you mention, as a question,
> Release candidate goes into staging repo first.
That's correct, for non-snapshot releases/release candidates. The business
with staging repo in Nexus is part of the `publish-release` step. I did not
change its basic logic, which is now in release-util.sh::maven_deploy():
1. Use mvn to set the version in pom.xml
1. Do `mvn deploy` to Nexus, using `-P apache-release,release` profiles,
which use the maven-release-plugin, nexus-staging-maven-plugin, and configs
from the HBase top-level pom and root ASF pom, to manage Nexus deployment.
1. Afterward (back to release-build.sh, at the end of `publish-release`
step), the code expects to find that the Staging Repo has been closed (made
immutable), and returns the repo ID to the user. Note that for snapshot
releases, the Staging Repo intermediate appears not to be used.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #1655: HBASE-24321 - Add writable MinVersions and read-only Scan to coproc S…
Apache9 commented on a change in pull request #1655:
URL: https://github.com/apache/hbase/pull/1655#discussion_r420529237
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanOptions.java
##
@@ -64,4 +66,13 @@ default void readAllVersions() {
void setKeepDeletedCells(KeepDeletedCells keepDeletedCells);
KeepDeletedCells getKeepDeletedCells();
+
+ int getMinVersions();
+
+ void setMinVersions(int minVersions);
+
+ /**
+ * Returns a read-only copy of the Scan object
Review comment:
The comment is a bit confusing as you are free to call the modification
methods of the returned Scan object? So it is not 'read only'.
Maybe better with "The returned Scan object is just a copy, modifying it
will have no effect, so treat it as read-only."
Of course I'm not a native English speaker, so feel free to use your own
word. Just show the meaning that 'you can modify, but no effect, so please do
not modify'.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] mattf-apache commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
mattf-apache commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420526164
##
File path: dev-support/create-release/release-build.sh
##
@@ -20,114 +20,180 @@
trap cleanup EXIT
# Source in utils.
-SELF=$(cd $(dirname $0) && pwd)
+SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=SCRIPTDIR/release-util.sh
. "$SELF/release-util.sh"
# Print usage and exit.
function exit_with_usage {
- cat << EOF
-Usage: release-build.sh
-Creates build deliverables from a tag/commit.
-Arguments:
- build Create binary packages and commit to
dist.apache.org/repos/dist/dev/hbase/
- publish-snapshot Publish snapshot release to Apache snapshots
- publish-release Publish a release to Apache release repo
-
-All other inputs are environment variables:
- GIT_REF - Release tag or commit to build from
- PACKAGE_VERSION - Release identifier in top level package directory (e.g.
2.1.2RC1)
- VERSION - (optional) Version of project being built (e.g. 2.1.2)
- ASF_USERNAME - Username of ASF committer account
- ASF_PASSWORD - Password of ASF committer account
- GPG_KEY - GPG key used to sign release artifacts
- GPG_PASSPHRASE - Passphrase for GPG key
- PROJECT - The project to build. No default.
-
-Set REPO environment to full path to repo to use
-to avoid re-downloading dependencies on each run.
+ cat <<'EOF'
+Usage: release-build.sh
+Creates release deliverables from a tag or commit.
+Argument: one of 'tag', 'publish-dist', 'publish-snapshot', or
'publish-release'
+ tag Prepares for release on specified git branch: Set release
version,
+update CHANGES and RELEASENOTES, create release tag,
+increment version for ongoing dev, and publish to Apache
git repo.
+ publish-dist Build and publish distribution packages (tarballs) to
Apache dist repo
+ publish-snapshot Build and publish maven artifacts snapshot release to
Apache snapshots repo
+ publish-release Build and publish maven artifacts release to Apache
release repo, and
+construct vote email from template
+
+All other inputs are environment variables. Please use do-release-docker.sh or
+do-release.sh to set up the needed environment variables. This script,
release-build.sh,
+is not intended to be called stand-alone, and such use is untested. The env
variables used are:
+
+Used for 'tag' and 'publish' stages:
+ PROJECT - The project to build. No default.
+ RELEASE_VERSION - Version used in pom files for release (e.g. 2.1.2)
+Required for 'tag'; defaults for 'publish' to the version in pom at GIT_REF
+ RELEASE_TAG - Name of release tag (e.g. 2.1.2RC0), also used by
+publish-dist as package version name in dist directory path
+ ASF_USERNAME - Username of ASF committer account
+ ASF_PASSWORD - Password of ASF committer account
+ DRY_RUN - 1:true (default), 0:false. If "1", does almost all the work, but
doesn't actually
+publish anything to upstream source or object repositories. It defaults to
"1", so if you want
+to actually publish you have to set '-f' (force) flag in do-release.sh or
do-release-docker.sh.
+
+Used only for 'tag':
+ GIT_NAME - Name to use with git
+ GIT_EMAIL - E-mail address to use with git
+ GIT_BRANCH - Git branch on which to make release. Tag is always placed at
HEAD of this branch.
+ NEXT_VERSION - Development version after release (e.g. 2.1.3-SNAPSHOT)
+
+Used only for 'publish':
+ GIT_REF - Release tag or commit to build from (defaults to $RELEASE_TAG;
only need to
+separately define GIT_REF if RELEASE_TAG is not actually present as a tag
at publish time)
+If both RELEASE_TAG and GIT_REF are undefined it will default to HEAD of
master.
+ GPG_KEY - GPG key id (usually email addr) used to sign release artifacts
+ GPG_PASSPHRASE - Passphrase for GPG key
+ REPO - Set to full path of a directory to use as maven local repo
(dependencies cache)
+to avoid re-downloading dependencies for each stage. It is automatically
set if you
+request full sequence of stages (tag, publish-dist, publish-release) in
do-release.sh.
For example:
- $ PROJECT="hbase-operator-tools" ASF_USERNAME=NAME ASF_PASSWORD=PASSWORD
GPG_PASSPHRASE=PASSWORD GPG_KEY=st...@apache.org ./release-build.sh build
+ $ PROJECT="hbase-operator-tools" ASF_USERNAME=NAME ASF_PASSWORD=PASSWORD
GPG_PASSPHRASE=PASSWORD GPG_KEY=st...@apache.org ./release-build.sh publish-dist
EOF
exit 1
}
set -e
function cleanup {
- echo "Cleaning up temp settings file." >&2
- rm "${tmp_settings}" &> /dev/null || true
# If REPO was set, then leave things be. Otherwise if we defined a repo
clean it out.
- if [[ -z "${REPO}" ]] && [[ -n "${tmp_repo}" ]]; then
-echo "Cleaning up temp repo in '${tmp_repo}'. set REPO to reuse
downloads." >&2
-rm -rf "${tmp_repo}" &> /dev/null || true
+ if [[ -z "${REPO}" ]] && [[ -n "${MAVEN_LOCAL_REPO}" ]]; the
[GitHub] [hbase] gjacoby126 commented on a change in pull request #1655: HBASE-24321 - Add writable MinVersions and read-only Scan to coproc S…
gjacoby126 commented on a change in pull request #1655:
URL: https://github.com/apache/hbase/pull/1655#discussion_r420527391
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanOptions.java
##
@@ -64,4 +66,10 @@ default void readAllVersions() {
void setKeepDeletedCells(KeepDeletedCells keepDeletedCells);
KeepDeletedCells getKeepDeletedCells();
+
+ int getMinVersions();
+
+ void setMinVersions(int minVersions);
+
+ Scan getScan() throws IOException;
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] mattf-apache commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
mattf-apache commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420526164
##
File path: dev-support/create-release/release-build.sh
##
@@ -20,114 +20,180 @@
trap cleanup EXIT
# Source in utils.
-SELF=$(cd $(dirname $0) && pwd)
+SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=SCRIPTDIR/release-util.sh
. "$SELF/release-util.sh"
# Print usage and exit.
function exit_with_usage {
- cat << EOF
-Usage: release-build.sh
-Creates build deliverables from a tag/commit.
-Arguments:
- build Create binary packages and commit to
dist.apache.org/repos/dist/dev/hbase/
- publish-snapshot Publish snapshot release to Apache snapshots
- publish-release Publish a release to Apache release repo
-
-All other inputs are environment variables:
- GIT_REF - Release tag or commit to build from
- PACKAGE_VERSION - Release identifier in top level package directory (e.g.
2.1.2RC1)
- VERSION - (optional) Version of project being built (e.g. 2.1.2)
- ASF_USERNAME - Username of ASF committer account
- ASF_PASSWORD - Password of ASF committer account
- GPG_KEY - GPG key used to sign release artifacts
- GPG_PASSPHRASE - Passphrase for GPG key
- PROJECT - The project to build. No default.
-
-Set REPO environment to full path to repo to use
-to avoid re-downloading dependencies on each run.
+ cat <<'EOF'
+Usage: release-build.sh
+Creates release deliverables from a tag or commit.
+Argument: one of 'tag', 'publish-dist', 'publish-snapshot', or
'publish-release'
+ tag Prepares for release on specified git branch: Set release
version,
+update CHANGES and RELEASENOTES, create release tag,
+increment version for ongoing dev, and publish to Apache
git repo.
+ publish-dist Build and publish distribution packages (tarballs) to
Apache dist repo
+ publish-snapshot Build and publish maven artifacts snapshot release to
Apache snapshots repo
+ publish-release Build and publish maven artifacts release to Apache
release repo, and
+construct vote email from template
+
+All other inputs are environment variables. Please use do-release-docker.sh or
+do-release.sh to set up the needed environment variables. This script,
release-build.sh,
+is not intended to be called stand-alone, and such use is untested. The env
variables used are:
+
+Used for 'tag' and 'publish' stages:
+ PROJECT - The project to build. No default.
+ RELEASE_VERSION - Version used in pom files for release (e.g. 2.1.2)
+Required for 'tag'; defaults for 'publish' to the version in pom at GIT_REF
+ RELEASE_TAG - Name of release tag (e.g. 2.1.2RC0), also used by
+publish-dist as package version name in dist directory path
+ ASF_USERNAME - Username of ASF committer account
+ ASF_PASSWORD - Password of ASF committer account
+ DRY_RUN - 1:true (default), 0:false. If "1", does almost all the work, but
doesn't actually
+publish anything to upstream source or object repositories. It defaults to
"1", so if you want
+to actually publish you have to set '-f' (force) flag in do-release.sh or
do-release-docker.sh.
+
+Used only for 'tag':
+ GIT_NAME - Name to use with git
+ GIT_EMAIL - E-mail address to use with git
+ GIT_BRANCH - Git branch on which to make release. Tag is always placed at
HEAD of this branch.
+ NEXT_VERSION - Development version after release (e.g. 2.1.3-SNAPSHOT)
+
+Used only for 'publish':
+ GIT_REF - Release tag or commit to build from (defaults to $RELEASE_TAG;
only need to
+separately define GIT_REF if RELEASE_TAG is not actually present as a tag
at publish time)
+If both RELEASE_TAG and GIT_REF are undefined it will default to HEAD of
master.
+ GPG_KEY - GPG key id (usually email addr) used to sign release artifacts
+ GPG_PASSPHRASE - Passphrase for GPG key
+ REPO - Set to full path of a directory to use as maven local repo
(dependencies cache)
+to avoid re-downloading dependencies for each stage. It is automatically
set if you
+request full sequence of stages (tag, publish-dist, publish-release) in
do-release.sh.
For example:
- $ PROJECT="hbase-operator-tools" ASF_USERNAME=NAME ASF_PASSWORD=PASSWORD
GPG_PASSPHRASE=PASSWORD GPG_KEY=st...@apache.org ./release-build.sh build
+ $ PROJECT="hbase-operator-tools" ASF_USERNAME=NAME ASF_PASSWORD=PASSWORD
GPG_PASSPHRASE=PASSWORD GPG_KEY=st...@apache.org ./release-build.sh publish-dist
EOF
exit 1
}
set -e
function cleanup {
- echo "Cleaning up temp settings file." >&2
- rm "${tmp_settings}" &> /dev/null || true
# If REPO was set, then leave things be. Otherwise if we defined a repo
clean it out.
- if [[ -z "${REPO}" ]] && [[ -n "${tmp_repo}" ]]; then
-echo "Cleaning up temp repo in '${tmp_repo}'. set REPO to reuse
downloads." >&2
-rm -rf "${tmp_repo}" &> /dev/null || true
+ if [[ -z "${REPO}" ]] && [[ -n "${MAVEN_LOCAL_REPO}" ]]; the
[GitHub] [hbase] Apache-HBase commented on pull request #1656: HBASE-24307 [Flakey Tests] krb server for secure thrift tests throws …
Apache-HBase commented on pull request #1656: URL: https://github.com/apache/hbase/pull/1656#issuecomment-624421167 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 32s | branch-2 passed | | +1 :green_heart: | compile | 2m 10s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 34s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 46s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 13s | the patch passed | | +1 :green_heart: | compile | 2m 8s | the patch passed | | +1 :green_heart: | javac | 2m 8s | the patch passed | | +1 :green_heart: | shadedjars | 4m 27s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 45s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 22s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 0m 47s | hbase-http in the patch passed. | | +1 :green_heart: | unit | 132m 27s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 4m 21s | hbase-thrift in the patch passed. | | | | 170m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1656 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 88b576edbd24 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 80561504d2 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/testReport/ | | Max. process+thread count | 3899 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-http hbase-server hbase-thrift U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1660: HBASE-24316 GCMulitpleMergedRegionsProcedure is not idempotent
Apache-HBase commented on pull request #1660: URL: https://github.com/apache/hbase/pull/1660#issuecomment-624420028 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 29s | master passed | | +1 :green_heart: | compile | 1m 59s | master passed | | +1 :green_heart: | shadedjars | 6m 46s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 32s | hbase-client in master failed. | | -0 :warning: | javadoc | 0m 52s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 22s | the patch passed | | +1 :green_heart: | compile | 1m 58s | the patch passed | | +1 :green_heart: | javac | 1m 58s | the patch passed | | +1 :green_heart: | shadedjars | 6m 43s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 31s | hbase-client in the patch failed. | | -0 :warning: | javadoc | 0m 47s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 29s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 196m 33s | hbase-server in the patch passed. | | | | 233m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1660/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1660 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 31893ad30579 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3d96007c37 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1660/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1660/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1660/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1660/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1660/1/testReport/ | | Max. process+thread count | 3115 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1660/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #1655: HBASE-24321 - Add writable MinVersions and read-only Scan to coproc S…
Apache9 commented on a change in pull request #1655:
URL: https://github.com/apache/hbase/pull/1655#discussion_r420522949
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java
##
@@ -685,6 +687,40 @@ public StoreFileReader
postStoreFileReaderOpen(ObserverContext ctx,
+Store store, ScanOptions options) throws IOException {
+if (options.getScan().getTimeRange().isAllTime()) {
Review comment:
OK, the rebuild of the ScanInfo is done in RegionCoprocessorHost, I used
to thought it is done in HStore on some other places.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1656: HBASE-24307 [Flakey Tests] krb server for secure thrift tests throws …
Apache-HBase commented on pull request #1656: URL: https://github.com/apache/hbase/pull/1656#issuecomment-624419193 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 1s | branch-2 passed | | +1 :green_heart: | compile | 2m 25s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 12s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 20s | hbase-common in branch-2 failed. | | -0 :warning: | javadoc | 0m 17s | hbase-http in branch-2 failed. | | -0 :warning: | javadoc | 0m 38s | hbase-server in branch-2 failed. | | -0 :warning: | javadoc | 0m 49s | hbase-thrift in branch-2 failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 52s | the patch passed | | +1 :green_heart: | compile | 2m 25s | the patch passed | | +1 :green_heart: | javac | 2m 25s | the patch passed | | +1 :green_heart: | shadedjars | 5m 8s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 17s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 17s | hbase-http in the patch failed. | | -0 :warning: | javadoc | 0m 38s | hbase-server in the patch failed. | | -0 :warning: | javadoc | 0m 51s | hbase-thrift in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 35s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 0m 41s | hbase-http in the patch passed. | | +1 :green_heart: | unit | 120m 28s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 3m 51s | hbase-thrift in the patch passed. | | | | 161m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1656 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 98eb228c2f03 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 80561504d2 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-http.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-http.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-thrift.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/testReport/ | | Max. process+thread count | 4464 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-http hbase-server hbase-thrift U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git S
[GitHub] [hbase] mattf-apache commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
mattf-apache commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420520956
##
File path: dev-support/create-release/release-build.sh
##
@@ -20,114 +20,180 @@
trap cleanup EXIT
# Source in utils.
-SELF=$(cd $(dirname $0) && pwd)
+SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=SCRIPTDIR/release-util.sh
. "$SELF/release-util.sh"
# Print usage and exit.
function exit_with_usage {
- cat << EOF
-Usage: release-build.sh
-Creates build deliverables from a tag/commit.
-Arguments:
- build Create binary packages and commit to
dist.apache.org/repos/dist/dev/hbase/
- publish-snapshot Publish snapshot release to Apache snapshots
- publish-release Publish a release to Apache release repo
-
-All other inputs are environment variables:
- GIT_REF - Release tag or commit to build from
- PACKAGE_VERSION - Release identifier in top level package directory (e.g.
2.1.2RC1)
- VERSION - (optional) Version of project being built (e.g. 2.1.2)
- ASF_USERNAME - Username of ASF committer account
- ASF_PASSWORD - Password of ASF committer account
- GPG_KEY - GPG key used to sign release artifacts
- GPG_PASSPHRASE - Passphrase for GPG key
- PROJECT - The project to build. No default.
-
-Set REPO environment to full path to repo to use
-to avoid re-downloading dependencies on each run.
+ cat <<'EOF'
+Usage: release-build.sh
+Creates release deliverables from a tag or commit.
+Argument: one of 'tag', 'publish-dist', 'publish-snapshot', or
'publish-release'
+ tag Prepares for release on specified git branch: Set release
version,
+update CHANGES and RELEASENOTES, create release tag,
+increment version for ongoing dev, and publish to Apache
git repo.
+ publish-dist Build and publish distribution packages (tarballs) to
Apache dist repo
+ publish-snapshot Build and publish maven artifacts snapshot release to
Apache snapshots repo
+ publish-release Build and publish maven artifacts release to Apache
release repo, and
+construct vote email from template
+
+All other inputs are environment variables. Please use do-release-docker.sh or
+do-release.sh to set up the needed environment variables. This script,
release-build.sh,
+is not intended to be called stand-alone, and such use is untested. The env
variables used are:
Review comment:
I was letting it go due to how specialized it is, but you're right;
there's no reason not to make it nice and clear. Will try to improve this; see
comment immediately above.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] gjacoby126 commented on a change in pull request #1655: HBASE-24321 - Add writable MinVersions and read-only Scan to coproc S…
gjacoby126 commented on a change in pull request #1655:
URL: https://github.com/apache/hbase/pull/1655#discussion_r420520899
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanInfo.java
##
@@ -174,8 +174,13 @@ public boolean isNewVersionBehavior() {
* Used for CP users for customizing max versions, ttl and keepDeletedCells.
*/
ScanInfo customize(int maxVersions, long ttl, KeepDeletedCells
keepDeletedCells) {
Review comment:
It was used by a few tests: TestCompaction, TestDefaultCompaction,
TestMajorCompaction.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] nyl3532016 commented on a change in pull request #1629: HBASE-24328 skip duplicate GCMultipleMergedRegionsProcedure while previous finished
nyl3532016 commented on a change in pull request #1629:
URL: https://github.com/apache/hbase/pull/1629#discussion_r420518956
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/GCMultipleMergedRegionsProcedure.java
##
@@ -78,6 +78,13 @@ protected Flow executeFromState(MasterProcedureEnv env,
GCMergedRegionsState sta
switch (state) {
case GC_MERGED_REGIONS_PREPARE:
// Nothing to do to prepare.
+ List parents = MetaTableAccessor.getMergeRegions(
+env.getMasterServices().getConnection(),
mergedChild.getRegionName());
+ if (parents == null || parents.isEmpty()) {
Review comment:
to check whether skip GCMultipleMergedRegionsProcedure
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] gjacoby126 commented on a change in pull request #1655: HBASE-24321 - Add writable MinVersions and read-only Scan to coproc S…
gjacoby126 commented on a change in pull request #1655:
URL: https://github.com/apache/hbase/pull/1655#discussion_r420520558
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java
##
@@ -685,6 +687,40 @@ public StoreFileReader
postStoreFileReaderOpen(ObserverContext ctx,
+Store store, ScanOptions options) throws IOException {
+if (options.getScan().getTimeRange().isAllTime()) {
Review comment:
See TestRegionCoprocessorHost. It's used to test that the scanner coproc
hooks are wired up correctly and can alter a ScanInfo.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] mattf-apache commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
mattf-apache commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420508959
##
File path: dev-support/create-release/release-build.sh
##
@@ -20,114 +20,180 @@
trap cleanup EXIT
# Source in utils.
-SELF=$(cd $(dirname $0) && pwd)
+SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=SCRIPTDIR/release-util.sh
. "$SELF/release-util.sh"
# Print usage and exit.
function exit_with_usage {
- cat << EOF
-Usage: release-build.sh
-Creates build deliverables from a tag/commit.
-Arguments:
- build Create binary packages and commit to
dist.apache.org/repos/dist/dev/hbase/
- publish-snapshot Publish snapshot release to Apache snapshots
- publish-release Publish a release to Apache release repo
-
-All other inputs are environment variables:
- GIT_REF - Release tag or commit to build from
- PACKAGE_VERSION - Release identifier in top level package directory (e.g.
2.1.2RC1)
- VERSION - (optional) Version of project being built (e.g. 2.1.2)
- ASF_USERNAME - Username of ASF committer account
- ASF_PASSWORD - Password of ASF committer account
- GPG_KEY - GPG key used to sign release artifacts
- GPG_PASSPHRASE - Passphrase for GPG key
- PROJECT - The project to build. No default.
-
-Set REPO environment to full path to repo to use
-to avoid re-downloading dependencies on each run.
+ cat <<'EOF'
+Usage: release-build.sh
+Creates release deliverables from a tag or commit.
+Argument: one of 'tag', 'publish-dist', 'publish-snapshot', or
'publish-release'
+ tag Prepares for release on specified git branch: Set release
version,
+update CHANGES and RELEASENOTES, create release tag,
+increment version for ongoing dev, and publish to Apache
git repo.
+ publish-dist Build and publish distribution packages (tarballs) to
Apache dist repo
+ publish-snapshot Build and publish maven artifacts snapshot release to
Apache snapshots repo
+ publish-release Build and publish maven artifacts release to Apache
release repo, and
Review comment:
What one should do to run a release candidate, is invoke
`do-release-docker.sh` (preferably) or `do-release.sh` (if there's a reason not
to use docker). I agree with your comments here and in the next item below,
that this script must clearly be documented as NOT the entrance script. And
yet, the many env variables that get passed around as inter-script
communication do need to be documented somewhere. Let me see if I can move
them to `do-release.sh`, and make the comments and the README more useful.
On a completely unrelated note, you mention, as a question,
> Release candidate goes into staging repo first.
The business with staging repo in Nexus is part of the `publish-release`
step. I did not change its basic logic, which is now in
release-util.sh::maven_deploy():
1. Use mvn to set the version in pom.xml
1. Do `mvn deploy` to Nexus, using `-P apache-release,release` profiles,
which use the maven-release-plugin, nexus-staging-maven-plugin, and configs
from the HBase top-level pom and root ASF pom, to manage Nexus deployment.
1. Afterward (back to release-build.sh, at the end of `publish-release`
step), the code expects to find that the Staging Repo has been closed, and
returns the repo ID to the user. Note that for snapshot releases, the Staging
Repo intermediate appears not to be used.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] nyl3532016 commented on a change in pull request #1629: HBASE-24328 skip duplicate GCMultipleMergedRegionsProcedure while previous finished
nyl3532016 commented on a change in pull request #1629:
URL: https://github.com/apache/hbase/pull/1629#discussion_r420518956
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/GCMultipleMergedRegionsProcedure.java
##
@@ -78,6 +78,13 @@ protected Flow executeFromState(MasterProcedureEnv env,
GCMergedRegionsState sta
switch (state) {
case GC_MERGED_REGIONS_PREPARE:
// Nothing to do to prepare.
+ List parents = MetaTableAccessor.getMergeRegions(
+env.getMasterServices().getConnection(),
mergedChild.getRegionName());
+ if (parents == null || parents.isEmpty()) {
Review comment:
to check whether skip GCMultipleMergedRegionsProcedure
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] mattf-apache commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
mattf-apache commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420508959
##
File path: dev-support/create-release/release-build.sh
##
@@ -20,114 +20,180 @@
trap cleanup EXIT
# Source in utils.
-SELF=$(cd $(dirname $0) && pwd)
+SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=SCRIPTDIR/release-util.sh
. "$SELF/release-util.sh"
# Print usage and exit.
function exit_with_usage {
- cat << EOF
-Usage: release-build.sh
-Creates build deliverables from a tag/commit.
-Arguments:
- build Create binary packages and commit to
dist.apache.org/repos/dist/dev/hbase/
- publish-snapshot Publish snapshot release to Apache snapshots
- publish-release Publish a release to Apache release repo
-
-All other inputs are environment variables:
- GIT_REF - Release tag or commit to build from
- PACKAGE_VERSION - Release identifier in top level package directory (e.g.
2.1.2RC1)
- VERSION - (optional) Version of project being built (e.g. 2.1.2)
- ASF_USERNAME - Username of ASF committer account
- ASF_PASSWORD - Password of ASF committer account
- GPG_KEY - GPG key used to sign release artifacts
- GPG_PASSPHRASE - Passphrase for GPG key
- PROJECT - The project to build. No default.
-
-Set REPO environment to full path to repo to use
-to avoid re-downloading dependencies on each run.
+ cat <<'EOF'
+Usage: release-build.sh
+Creates release deliverables from a tag or commit.
+Argument: one of 'tag', 'publish-dist', 'publish-snapshot', or
'publish-release'
+ tag Prepares for release on specified git branch: Set release
version,
+update CHANGES and RELEASENOTES, create release tag,
+increment version for ongoing dev, and publish to Apache
git repo.
+ publish-dist Build and publish distribution packages (tarballs) to
Apache dist repo
+ publish-snapshot Build and publish maven artifacts snapshot release to
Apache snapshots repo
+ publish-release Build and publish maven artifacts release to Apache
release repo, and
Review comment:
What one should do to run a release candidate, is invoke
`do-release-docker.sh` (preferably) or `do-release.sh` (if there's a reason not
to use docker). I agree with your comments here and in the next item below,
that this script must clearly be documented as NOT the entrance script. And
yet, the many env variables that get passed around as inter-script
communication do need to be documented somewhere. Let me see if I can move
them to `do-release.sh`, and make the comments and the README more useful.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1661: Backport: HBASE-24273 HBCK's "Orphan Regions on FileSystem" reports regions wit…
Apache-HBase commented on pull request #1661: URL: https://github.com/apache/hbase/pull/1661#issuecomment-624415634 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 50s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 30s | branch-2 passed | | +1 :green_heart: | compile | 1m 5s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 25s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 41s | hbase-server in branch-2 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 9s | the patch passed | | +1 :green_heart: | compile | 1m 6s | the patch passed | | +1 :green_heart: | javac | 1m 6s | the patch passed | | +1 :green_heart: | shadedjars | 6m 18s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 49s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 135m 41s | hbase-server in the patch passed. | | | | 162m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1661/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1661 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux cbd6fc9a471d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 80561504d2 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1661/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1661/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1661/1/testReport/ | | Max. process+thread count | 3496 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1661/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] binlijin commented on a change in pull request #1588: HBASE-24262 Improvement update_config for PressureAwareCompactionThroughputController
binlijin commented on a change in pull request #1588:
URL: https://github.com/apache/hbase/pull/1588#discussion_r420518128
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/throttle/CompactionThroughputControllerFactory.java
##
@@ -47,24 +53,35 @@ private CompactionThroughputControllerFactory() {
public static ThroughputController create(RegionServerServices server,
Configuration conf) {
-Class clazz =
getThroughputControllerClass(conf);
-ThroughputController controller = ReflectionUtils.newInstance(clazz, conf);
-controller.setup(server);
+Class clazz =
+getThroughputControllerClass(conf);
+ThroughputController controller = controllerMap.get(clazz);
+if (controller == null) {
+ controller = ReflectionUtils.newInstance(clazz, conf);
+ controller.setup(server);
+ controllerMap.put(clazz, controller);
+} else {
Review comment:
why change it into this?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/OffPeakHours.java
##
@@ -87,6 +87,7 @@ private static boolean isValidHour(int hour) {
OffPeakHoursImpl(int startHour, int endHour) {
this.startHour = startHour;
this.endHour = endHour;
+ LOG.info(this.toString());
Review comment:
The log is not good, do not have info about the region.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] nyl3532016 edited a comment on pull request #1629: HBASE-24328 skip duplicate GCMultipleMergedRegionsProcedure while previous finished
nyl3532016 edited a comment on pull request #1629: URL: https://github.com/apache/hbase/pull/1629#issuecomment-624068135 @saintstack could you help review this one ? This patch is related to HBASE-24250 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1628: HBASE-24304 Separate a hbase-asyncfs module
Apache-HBase commented on pull request #1628: URL: https://github.com/apache/hbase/pull/1628#issuecomment-624412558 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 56s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 45s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 22s | master passed | | +1 :green_heart: | checkstyle | 2m 44s | master passed | | +1 :green_heart: | spotbugs | 18m 56s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 30s | the patch passed | | -0 :warning: | checkstyle | 2m 39s | root: The patch generated 1 new + 523 unchanged - 5 fixed = 524 total (was 528) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 23s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 12m 40s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 16m 16s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 2m 17s | The patch does not generate ASF License warnings. | | | | 76m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/7/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1628 | | Optional Tests | dupname asflicense xml hadoopcheck spotbugs hbaseanti checkstyle | | uname | Linux d21f9adba64a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3d96007c37 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/7/artifact/yetus-general-check/output/diff-checkstyle-root.txt | | Max. process+thread count | 139 (vs. ulimit of 12500) | | modules | C: hbase-asyncfs . hbase-assembly hbase-endpoint hbase-examples hbase-mapreduce hbase-rest hbase-server hbase-shaded/hbase-shaded-testing-util hbase-testing-util hbase-thrift U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24322) UnsafeAvailChecker should also check that required methods are available
[
https://issues.apache.org/jira/browse/HBASE-24322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100406#comment-17100406
]
Hudson commented on HBASE-24322:
Results for branch branch-2
[build #2644 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2644/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2644/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2644/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2644/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2644/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> UnsafeAvailChecker should also check that required methods are available
>
>
> Key: HBASE-24322
> URL: https://issues.apache.org/jira/browse/HBASE-24322
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 2.1.10, 2.2.5
>
>
> We had a weird test failure due to accidentally running tests with some Java
> > 8 (exact version I am unsure of), where Unsafe is available, but the method
> signatures were different or some methods were removed, leading to this:
> {noformat}
> 020-05-02 14:57:15,145 ERROR [main] master.HMasterCommandLine: Master exiting
> java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:143)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:237)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:163)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:225)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:138)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2911)
> Caused by: java.lang.NoSuchMethodError: 'void
> sun.misc.Unsafe.putInt(java.lang.Object, int, int)'
> at org.apache.hadoop.hbase.util.UnsafeAccess.putInt(UnsafeAccess.java:233)
> at
> org.apache.hadoop.hbase.util.Bytes$ConverterHolder$UnsafeConverter.putInt(Bytes.java:1499)
> at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:1021)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.appendMetaData(RecoverableZooKeeper.java:850)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:640)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:1027)
> at
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.setMasterAddress(MasterAddressTracker.java:211)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2095)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:520)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.(HMasterCommandLine.java:315)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at
> java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:138)
> ... 7 more
> {noformat}
> We should also check that all methods that will be invoked on Unsafe in
> UnsafeAccess.java are available when deciding in UnsafeAvailChecker if Unsafe
> is available (and usable).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24311) Add more details in MultiVersionConcurrencyControl STUCK log message
[
https://issues.apache.org/jira/browse/HBASE-24311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100404#comment-17100404
]
Hudson commented on HBASE-24311:
Results for branch branch-2
[build #2644 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2644/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2644/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2644/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2644/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2644/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add more details in MultiVersionConcurrencyControl STUCK log message
>
>
> Key: HBASE-24311
> URL: https://issues.apache.org/jira/browse/HBASE-24311
> Project: HBase
> Issue Type: Improvement
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0
>
>
> Now the warn logs look like
> {code}
> STUCK: MultiVersionConcurrencyControl{readPoint=30944485, writePoint=30944498}
> {code}
> We don't have any details on which region the writes are stuck. It would be
> better to include the region name detail also in this log.
> May be we can even log for how long we are waiting for the read point to
> catch up.
> cc [~stack]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24314) Some classes still use log4j logger directly
[
https://issues.apache.org/jira/browse/HBASE-24314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100405#comment-17100405
]
Hudson commented on HBASE-24314:
Results for branch branch-2
[build #2644 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2644/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2644/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2644/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2644/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2644/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Some classes still use log4j logger directly
>
>
> Key: HBASE-24314
> URL: https://issues.apache.org/jira/browse/HBASE-24314
> Project: HBase
> Issue Type: Bug
> Components: logging
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0-alpha-1, 2.3.0, 2.2.5
>
>
> Found this when implementing HBASE-24309. Should use slf4j instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1593: HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry …
Apache-HBase commented on pull request #1593: URL: https://github.com/apache/hbase/pull/1593#issuecomment-624407067 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 37s | master passed | | +1 :green_heart: | checkstyle | 1m 55s | master passed | | +1 :green_heart: | spotbugs | 3m 45s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 27s | the patch passed | | +1 :green_heart: | checkstyle | 1m 50s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 47s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 5m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 37s | The patch does not generate ASF License warnings. | | | | 43m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/6/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1593 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux b456a0c97f8b 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3d96007c37 | | Max. process+thread count | 95 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/6/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] mattf-apache commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
mattf-apache commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420508959
##
File path: dev-support/create-release/release-build.sh
##
@@ -20,114 +20,180 @@
trap cleanup EXIT
# Source in utils.
-SELF=$(cd $(dirname $0) && pwd)
+SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=SCRIPTDIR/release-util.sh
. "$SELF/release-util.sh"
# Print usage and exit.
function exit_with_usage {
- cat << EOF
-Usage: release-build.sh
-Creates build deliverables from a tag/commit.
-Arguments:
- build Create binary packages and commit to
dist.apache.org/repos/dist/dev/hbase/
- publish-snapshot Publish snapshot release to Apache snapshots
- publish-release Publish a release to Apache release repo
-
-All other inputs are environment variables:
- GIT_REF - Release tag or commit to build from
- PACKAGE_VERSION - Release identifier in top level package directory (e.g.
2.1.2RC1)
- VERSION - (optional) Version of project being built (e.g. 2.1.2)
- ASF_USERNAME - Username of ASF committer account
- ASF_PASSWORD - Password of ASF committer account
- GPG_KEY - GPG key used to sign release artifacts
- GPG_PASSPHRASE - Passphrase for GPG key
- PROJECT - The project to build. No default.
-
-Set REPO environment to full path to repo to use
-to avoid re-downloading dependencies on each run.
+ cat <<'EOF'
+Usage: release-build.sh
+Creates release deliverables from a tag or commit.
+Argument: one of 'tag', 'publish-dist', 'publish-snapshot', or
'publish-release'
+ tag Prepares for release on specified git branch: Set release
version,
+update CHANGES and RELEASENOTES, create release tag,
+increment version for ongoing dev, and publish to Apache
git repo.
+ publish-dist Build and publish distribution packages (tarballs) to
Apache dist repo
+ publish-snapshot Build and publish maven artifacts snapshot release to
Apache snapshots repo
+ publish-release Build and publish maven artifacts release to Apache
release repo, and
Review comment:
If you don't specify a step, it will do `tag`, `publish-dist` (formerly
misnamed as "build"), and `publish-release`, in that order.
The business with staging repo in Nexus is part of the `publish-release`
step. I did not change its basic logic, which is now in
release-util.sh::maven_deploy():
1. Use mvn to set the version in pom.xml
1. Do `mvn deploy` to Nexus, using `-P apache-release,release` profiles,
which use the maven-release-plugin, nexus-staging-maven-plugin, and configs
from the HBase top-level pom and root ASF pom, to manage Nexus deployment.
1. Afterward (back to release-build.sh, at the end of `publish-release`
step), the code expects to find that the Staging Repo has been closed, and
returns the repo ID to the user. Note that for snapshot releases, the Staging
Repo intermediate appears not to be used.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1660: HBASE-24316 GCMulitpleMergedRegionsProcedure is not idempotent
Apache-HBase commented on pull request #1660: URL: https://github.com/apache/hbase/pull/1660#issuecomment-624405374 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 38s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 48s | master passed | | +1 :green_heart: | compile | 1m 45s | master passed | | +1 :green_heart: | shadedjars | 6m 45s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 8s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 39s | the patch passed | | +1 :green_heart: | compile | 1m 21s | the patch passed | | +1 :green_heart: | javac | 1m 21s | the patch passed | | +1 :green_heart: | shadedjars | 5m 3s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 3s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 134m 41s | hbase-server in the patch passed. | | | | 165m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1660/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1660 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 78a02ba4344f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3d96007c37 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1660/1/testReport/ | | Max. process+thread count | 4447 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1660/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1659: HBASE-24332: TestJMXListener.setupBeforeClass can fail due to not get…
Apache-HBase commented on pull request #1659: URL: https://github.com/apache/hbase/pull/1659#issuecomment-624405179 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 9m 47s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 14s | branch-2 passed | | +1 :green_heart: | compile | 1m 46s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 15s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 53s | the patch passed | | +1 :green_heart: | compile | 1m 45s | the patch passed | | +1 :green_heart: | javac | 1m 45s | the patch passed | | +1 :green_heart: | shadedjars | 5m 7s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 37s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 9s | hbase-http in the patch passed. | | -1 :x: | unit | 225m 1s | hbase-server in the patch failed. | | | | 264m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1659/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1659 | | JIRA Issue | HBASE-24332 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 49b4ff0293a6 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 5dc10dbba5 | | Default Java | 1.8.0_232 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1659/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1659/1/testReport/ | | Max. process+thread count | 2413 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-http hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1659/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1659: HBASE-24332: TestJMXListener.setupBeforeClass can fail due to not get…
Apache-HBase commented on pull request #1659: URL: https://github.com/apache/hbase/pull/1659#issuecomment-624403494 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 9m 47s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 4s | branch-2 passed | | +1 :green_heart: | compile | 2m 9s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 15s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 18s | hbase-common in branch-2 failed. | | -0 :warning: | javadoc | 0m 22s | hbase-http in branch-2 failed. | | -0 :warning: | javadoc | 0m 43s | hbase-server in branch-2 failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 41s | the patch passed | | +1 :green_heart: | compile | 2m 1s | the patch passed | | +1 :green_heart: | javac | 2m 1s | the patch passed | | +1 :green_heart: | shadedjars | 6m 15s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 17s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 18s | hbase-http in the patch failed. | | -0 :warning: | javadoc | 0m 52s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 20s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 2s | hbase-http in the patch passed. | | -1 :x: | unit | 211m 22s | hbase-server in the patch failed. | | | | 256m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1659/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1659 | | JIRA Issue | HBASE-24332 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5c10d3baf882 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 5dc10dbba5 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1659/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1659/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-http.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1659/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1659/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1659/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-http.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1659/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1659/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1659/1/testReport/ | | Max. process+thread count | 2534 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-http hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1659/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] mattf-apache commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
mattf-apache commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420503750
##
File path: dev-support/create-release/do-release.sh
##
@@ -66,26 +82,34 @@ function should_build {
fi
}
-if should_build "tag" && [ $SKIP_TAG = 0 ]; then
+if should_build "tag" && [ "$SKIP_TAG" = 0 ]; then
run_silent "Creating release tag $RELEASE_TAG..." "tag.log" \
-"$SELF/release-tag.sh"
- echo "It may take some time for the tag to be synchronized to github."
- echo "Press enter when you've verified that the new tag ($RELEASE_TAG) is
available."
- read
+"$SELF/release-build.sh" tag
+ if is_dry_run; then
+export TAG_SAME_DRY_RUN="true";
+ else
+echo "It may take some time for the tag to be synchronized to github."
+echo "Press enter when you've verified that the new tag ($RELEASE_TAG) is
available."
+read -r
+ fi
Review comment:
Apparently it was in the original code swiped from the Spark project. I
think it would be pretty easy to replace it with a little wait loop; I'll do
that.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] mattf-apache commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
mattf-apache commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r420502419
##
File path: dev-support/create-release/do-release.sh
##
@@ -66,26 +82,34 @@ function should_build {
fi
}
-if should_build "tag" && [ $SKIP_TAG = 0 ]; then
+if should_build "tag" && [ "$SKIP_TAG" = 0 ]; then
run_silent "Creating release tag $RELEASE_TAG..." "tag.log" \
-"$SELF/release-tag.sh"
- echo "It may take some time for the tag to be synchronized to github."
- echo "Press enter when you've verified that the new tag ($RELEASE_TAG) is
available."
- read
+"$SELF/release-build.sh" tag
+ if is_dry_run; then
+export TAG_SAME_DRY_RUN="true";
+ else
+echo "It may take some time for the tag to be synchronized to github."
+echo "Press enter when you've verified that the new tag ($RELEASE_TAG) is
available."
+read -r
+ fi
else
echo "Skipping tag creation for $RELEASE_TAG."
fi
-if should_build "build"; then
- run_silent "Building ${PROJECT}..." "build.log" \
-"$SELF/release-build.sh" build
+if should_build "publish-dist"; then
+ run_silent "Publishing distribution packages (tarballs)" "publish-dist.log" \
+"$SELF/release-build.sh" publish-dist
else
- echo "Skipping build step."
+ echo "Skipping publish-dist step."
Review comment:
Done, in Release Note in jira (along with the reason why).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-24318) Create-release scripts fixes and enhancements
[ https://issues.apache.org/jira/browse/HBASE-24318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Foley updated HBASE-24318: -- Release Note: Release Managers will want to take note of two changes in the create-release tools: - The names of the release steps are now `tag`, `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two being mutually exclusive alternatives). `Publish-dist` was formerly just named `build`, which disguised the fact that it built AND published tarballs to https://dist.apache.org/repos/dist/release/ , and that publish-snapshot/publish-release do their own build before publishing maven artifacts to the Nexus repos. - "Dry run" has become the _default_ behavior. Doing a dry run first decreases the likelihood of failure part-way through, which could result in inconsistent artifacts in the repos. To actually do the release, add the `-f` (force) option flag when you invoke `do-release-docker.sh` or `do-release.sh`. This is similar to `-f` usage in some Linux and git commands that have high cost for mistakes. This is a change from the former `-n` to turn ON dry run. > Create-release scripts fixes and enhancements > - > > Key: HBASE-24318 > URL: https://issues.apache.org/jira/browse/HBASE-24318 > Project: HBase > Issue Type: Improvement > Components: create-release >Affects Versions: 3.0.0-alpha-1, 2.3.0 > Environment: Linux Docker container, or Mac OS X without Docker >Reporter: Matthew Foley >Assignee: Matthew Foley >Priority: Major > > The create-release tools are a set of scripts that promote best practice in > making Apache releases, by automating most of the steps and running them > under Docker for reliability. However, the current state of the scripts has > many bugs and ambiguities. > The proposed PR cleans up the code and clarifies usage. It enables: > - Clear statement of the four steps, which are now called `tag`, > `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two > being mutually exclusive alternatives). > - Ability to do the three tag-dist-release steps with a single command, or do > any of the steps singly. (Running singly had bugs and unfulfilled > dependencies before.) > - Ability to do a reliable and useful "dry run" of all steps or each step, > and chain together the tag step with publish steps in a dry run. > - Ability to run any or all steps correctly in Docker or outside of Docker, > on Linux and Mac. > - Cleaned up all `shellcheck` errors in the scripts, and removed ambiguities > and redundancies in the many environment variables used. > In addition, the changes move the code toward being more general / less > HBase-specific, so it can be run on any Apache project (while still > accommodating HBase-specific features regarding how sub-projects are named > and organized in Jira and release repos). In future I propose to take it > further along that path, and move create-release into Yetus (recognizing that > this create-release code has been passed between a couple other projects > already). > These changes have NO IMPACT on HBase functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] mattf-apache commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
mattf-apache commented on a change in pull request #1643: URL: https://github.com/apache/hbase/pull/1643#discussion_r420502004 ## File path: dev-support/create-release/do-release-docker.sh ## @@ -64,24 +65,26 @@ This script runs the release scripts inside a docker image. Options: -d [path]required. working directory. output will be written to "output" in here. - -n dry run mode. Checks and local builds, but does not upload anything. + -f "force" -- actually publish this release. Unless you specify '-f', it will + default to dry run mode, which checks and does local builds, but does not upload anything. Review comment: Great idea, added a Release Note in the jira. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #1593: HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry …
Apache9 commented on pull request #1593: URL: https://github.com/apache/hbase/pull/1593#issuecomment-624396591 > Ya, my question was about the word "retries". That will show up in the exception message (IIUC). These give a false impression that things are being retried, right (when in reality they are hedged, in some sense). I think query multiple masters could also be some types of retry? If this master is not available, we just try another, and if all are failed we will return the exception to you. > I was thinking about Call#setException(), which cleans up the caller state and propagates the exception to the the future callback? For example, if a master is hung and the RPC is hung, that state would be cleaned up quicker. Not a big deal? Only a small amount of memory. Implementing this will lead to a more complicated logic, we have to reference the CompletableFutures for other requests and call cancel on them, and we also need to deal with the cancel event and do the clean up work. Comparing to the benefits, I do not think it worths... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Work started] (HBASE-24318) Create-release scripts fixes and enhancements
[ https://issues.apache.org/jira/browse/HBASE-24318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-24318 started by Matthew Foley. - > Create-release scripts fixes and enhancements > - > > Key: HBASE-24318 > URL: https://issues.apache.org/jira/browse/HBASE-24318 > Project: HBase > Issue Type: Improvement > Components: create-release >Affects Versions: 3.0.0-alpha-1, 2.3.0 > Environment: Linux Docker container, or Mac OS X without Docker >Reporter: Matthew Foley >Assignee: Matthew Foley >Priority: Major > > The create-release tools are a set of scripts that promote best practice in > making Apache releases, by automating most of the steps and running them > under Docker for reliability. However, the current state of the scripts has > many bugs and ambiguities. > The proposed PR cleans up the code and clarifies usage. It enables: > - Clear statement of the four steps, which are now called `tag`, > `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two > being mutually exclusive alternatives). > - Ability to do the three tag-dist-release steps with a single command, or do > any of the steps singly. (Running singly had bugs and unfulfilled > dependencies before.) > - Ability to do a reliable and useful "dry run" of all steps or each step, > and chain together the tag step with publish steps in a dry run. > - Ability to run any or all steps correctly in Docker or outside of Docker, > on Linux and Mac. > - Cleaned up all `shellcheck` errors in the scripts, and removed ambiguities > and redundancies in the many environment variables used. > In addition, the changes move the code toward being more general / less > HBase-specific, so it can be run on any Apache project (while still > accommodating HBase-specific features regarding how sub-projects are named > and organized in Jira and release repos). In future I propose to take it > further along that path, and move create-release into Yetus (recognizing that > this create-release code has been passed between a couple other projects > already). > These changes have NO IMPACT on HBase functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1628: HBASE-24304 Separate a hbase-asyncfs module
Apache-HBase commented on pull request #1628: URL: https://github.com/apache/hbase/pull/1628#issuecomment-624393175 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1656: HBASE-24307 [Flakey Tests] krb server for secure thrift tests throws …
Apache-HBase commented on pull request #1656: URL: https://github.com/apache/hbase/pull/1656#issuecomment-624393106 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 38s | branch-2 passed | | +1 :green_heart: | checkstyle | 2m 29s | branch-2 passed | | +1 :green_heart: | spotbugs | 4m 26s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 5s | the patch passed | | +1 :green_heart: | checkstyle | 0m 23s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 0m 14s | hbase-http: The patch generated 0 new + 8 unchanged - 1 fixed = 8 total (was 9) | | -0 :warning: | checkstyle | 1m 12s | hbase-server: The patch generated 1 new + 217 unchanged - 1 fixed = 218 total (was 218) | | +1 :green_heart: | checkstyle | 0m 40s | hbase-thrift: The patch generated 0 new + 0 unchanged - 4 fixed = 0 total (was 4) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 11m 20s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 5m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 50s | The patch does not generate ASF License warnings. | | | | 42m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1656 | | Optional Tests | dupname asflicense hadoopcheck xml spotbugs hbaseanti checkstyle | | uname | Linux 93d42e7eaeb4 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 80561504d2 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-http hbase-server hbase-thrift U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1662: Backport: HBASE-24273 HBCK's "Orphan Regions on FileSystem" reports regions wit…
Apache-HBase commented on pull request #1662: URL: https://github.com/apache/hbase/pull/1662#issuecomment-624390734 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 54s | branch-2.3 passed | | +1 :green_heart: | checkstyle | 1m 16s | branch-2.3 passed | | +1 :green_heart: | spotbugs | 2m 5s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 30s | the patch passed | | -0 :warning: | checkstyle | 1m 14s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 42s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 13s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 41m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1662/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1662 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux f5f193971457 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / bee66bd22a | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1662/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1662/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #1655: HBASE-24321 - Add writable MinVersions and read-only Scan to coproc S…
Apache9 commented on a change in pull request #1655:
URL: https://github.com/apache/hbase/pull/1655#discussion_r420489073
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java
##
@@ -685,6 +687,40 @@ public StoreFileReader
postStoreFileReaderOpen(ObserverContext ctx,
+Store store, ScanOptions options) throws IOException {
+if (options.getScan().getTimeRange().isAllTime()) {
Review comment:
I think this class is only used to test whether a specific CP method has
been called? What is this used for?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanInfo.java
##
@@ -174,8 +174,13 @@ public boolean isNewVersionBehavior() {
* Used for CP users for customizing max versions, ttl and keepDeletedCells.
*/
ScanInfo customize(int maxVersions, long ttl, KeepDeletedCells
keepDeletedCells) {
Review comment:
Is this method still referenced? Just remove it?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CustomizedScanInfoBuilder.java
##
@@ -34,8 +36,18 @@
private KeepDeletedCells keepDeletedCells = null;
+ private Integer minVersions;
+
+ private final Scan scan;
+
public CustomizedScanInfoBuilder(ScanInfo scanInfo) {
this.scanInfo = scanInfo;
+this.scan = new Scan();
+ }
+ public CustomizedScanInfoBuilder(ScanInfo scanInfo, Scan scan) throws
IOException {
Review comment:
Let's just catch the IOException in the code here and then throw an
AssertionError to indicate this should not happen, and we can filed another
issue to remove the IOException from throws.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanOptions.java
##
@@ -64,4 +66,10 @@ default void readAllVersions() {
void setKeepDeletedCells(KeepDeletedCells keepDeletedCells);
KeepDeletedCells getKeepDeletedCells();
+
+ int getMinVersions();
+
+ void setMinVersions(int minVersions);
+
+ Scan getScan() throws IOException;
Review comment:
Please add a comment to say that the returned Scan object is only for
reading, changing it will have no effect to the actual scan operation.
And please modify the javadoc of this class to indicate that now we can also
change min versions?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1655: HBASE-24321 - Add writable MinVersions and read-only Scan to coproc S…
Apache-HBase commented on pull request #1655: URL: https://github.com/apache/hbase/pull/1655#issuecomment-624383024 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 57s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | shadedjars | 5m 34s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 43s | the patch passed | | +1 :green_heart: | compile | 0m 57s | the patch passed | | +1 :green_heart: | javac | 0m 57s | the patch passed | | +1 :green_heart: | shadedjars | 5m 25s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 34s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 194m 22s | hbase-server in the patch passed. | | | | 219m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1655/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1655 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 884c3c5c9459 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 404849ab36 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1655/2/testReport/ | | Max. process+thread count | 3285 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1655/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1661: Backport: HBASE-24273 HBCK's "Orphan Regions on FileSystem" reports regions wit…
Apache-HBase commented on pull request #1661: URL: https://github.com/apache/hbase/pull/1661#issuecomment-624382457 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 35s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 9s | branch-2 passed | | +1 :green_heart: | spotbugs | 1m 57s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 8s | the patch passed | | -0 :warning: | checkstyle | 1m 6s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 43s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 36m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1661/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1661 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux bef15e21c415 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 80561504d2 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1661/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1661/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24211) Create table is slow in large cluster when AccessController is enabled.
[
https://issues.apache.org/jira/browse/HBASE-24211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100352#comment-17100352
]
HBase QA commented on HBASE-24211:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 7m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
8s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} branch-1 passed with JDK v1.8.0_252 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} branch-1 passed with JDK v1.7.0_262 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
13s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
8s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} branch-1 passed with JDK v1.8.0_252 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} branch-1 passed with JDK v1.7.0_262 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 2m
49s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
20s{color} | {color:green} branch-1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed with JDK v1.8.0_252 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed with JDK v1.7.0_262 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} hbase-client: The patch generated 1 new + 60 unchanged
- 0 fixed = 61 total (was 60) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
56s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
5m 22s{color} | {color:green} Patch does not cause any errors with Hadoop 2.8.5
2.9.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed with JDK v1.8.0_252 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} the patch passed with JDK v1.7.0_262 {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{colo
[GitHub] [hbase] Apache-HBase commented on pull request #1658: HBASE-24211: [Addendum] Create table is slow in large cluster when AccessController is enabled.
Apache-HBase commented on pull request #1658: URL: https://github.com/apache/hbase/pull/1658#issuecomment-624377740 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 7m 20s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +0 :ok: | mvndep | 1m 25s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 8m 8s | branch-1 passed | | +1 :green_heart: | compile | 1m 2s | branch-1 passed with JDK v1.8.0_252 | | +1 :green_heart: | compile | 1m 7s | branch-1 passed with JDK v1.7.0_262 | | +1 :green_heart: | checkstyle | 2m 13s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 8s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | branch-1 passed with JDK v1.8.0_252 | | +1 :green_heart: | javadoc | 1m 6s | branch-1 passed with JDK v1.7.0_262 | | +0 :ok: | spotbugs | 2m 49s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 20s | branch-1 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 3s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed with JDK v1.8.0_252 | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed with JDK v1.7.0_262 | | +1 :green_heart: | javac | 1m 4s | the patch passed | | -1 :x: | checkstyle | 0m 36s | hbase-client: The patch generated 1 new + 60 unchanged - 0 fixed = 61 total (was 60) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 56s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 5m 22s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 1m 1s | the patch passed with JDK v1.8.0_252 | | +1 :green_heart: | javadoc | 1m 12s | the patch passed with JDK v1.7.0_262 | | +1 :green_heart: | findbugs | 4m 29s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 41s | hbase-client in the patch passed. | | -1 :x: | unit | 135m 49s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 1m 5s | The patch does not generate ASF License warnings. | | | | 195m 59s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.util.TestHBaseFsck | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1658/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1658 | | JIRA Issue | HBASE-24211 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 072dd1a7ebf7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1658/out/precommit/personality/provided.sh | | git revision | branch-1 / e4dbfe9 | | Default Java | 1.7.0_262 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_252 /usr/lib/jvm/zulu-7-amd64:1.7.0_262 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1658/1/artifact/out/diff-checkstyle-hbase-client.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1658/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1658/1/testReport/ | | Max. process+thread count | 4350 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1658/1/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetu
[GitHub] [hbase] Apache-HBase commented on pull request #1656: HBASE-24307 [Flakey Tests] krb server for secure thrift tests throws …
Apache-HBase commented on pull request #1656: URL: https://github.com/apache/hbase/pull/1656#issuecomment-624377639 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 48s | branch-2 passed | | +1 :green_heart: | compile | 2m 7s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 57s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 42s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 32s | the patch passed | | +1 :green_heart: | compile | 2m 9s | the patch passed | | +1 :green_heart: | javac | 2m 9s | the patch passed | | +1 :green_heart: | shadedjars | 4m 52s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 23s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 7s | hbase-http in the patch passed. | | +1 :green_heart: | unit | 197m 33s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 5m 42s | hbase-thrift in the patch passed. | | | | 236m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1656 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux fd6772b6c744 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 5dc10dbba5 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/2/testReport/ | | Max. process+thread count | 2599 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-http hbase-server hbase-thrift U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Reopened] (HBASE-24273) HBCK's "Orphan Regions on FileSystem" reports regions with referenced HFiles
[ https://issues.apache.org/jira/browse/HBASE-24273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack reopened HBASE-24273: --- Whoop. Premature close by me. Reopening. Close when backports done. Sorry about that [~huaxiangsun] > HBCK's "Orphan Regions on FileSystem" reports regions with referenced HFiles > > > Key: HBASE-24273 > URL: https://issues.apache.org/jira/browse/HBASE-24273 > Project: HBase > Issue Type: Bug > Components: hbck2 >Affects Versions: 2.2.4 > Environment: HBase 2.2.4 > Hadoop 3.1.3 >Reporter: Andrey Elenskiy >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.3.0 > > > This issue came up after merging regions. MergeTableRegionsProcedure removes > the parent regions from hbase:meta and creates HFile references in child > region to the old parent regions. Running `hbck_chore_run` right after the > `merge_region` will show the parent regions in "Orphan Regions on FileSystem" > until major compaction is run on child region which will remove HFile > references and cause Catalog Janitor to clean up the parent regions. > There are probably other situations which can cause the same issue (maybe > region split?) > Having "Orphan Regions on FileSystem" list parent regions and suggest to > "_hbase completebulkload_" is dangerous in this case as completing bulk load > will lead to stale HFile references in child region which will cause its OPEN > to fail because referenced HFile doesn't exist. > Figuring out these things for database administrators is tedious, so I think > it would be reasonable to not consider regions with referenced HFiles to be > orphans (or maybe could give an extra hint saying that it has referenced > HFiles). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-24069) Extend HBASE-16209 strategy (Provide an ExponentialBackOffPolicy sleep between failed region open requests) to region close and split requests
[ https://issues.apache.org/jira/browse/HBASE-24069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Guggilam reassigned HBASE-24069: Assignee: Sandeep Guggilam > Extend HBASE-16209 strategy (Provide an ExponentialBackOffPolicy sleep > between failed region open requests) to region close and split requests > -- > > Key: HBASE-24069 > URL: https://issues.apache.org/jira/browse/HBASE-24069 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Affects Versions: 1.6.0 >Reporter: Andrew Kyle Purtell >Assignee: Sandeep Guggilam >Priority: Major > Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0 > > > In HBASE-16209 we provide an ExponentialBackOffPolicy sleep between failed > region open requests. This should be extended to also apply to region close > and split requests. Will reduce the likelihood of FAILED_CLOSE transitions in > production by being more tolerant of temporary regionserver loading issues, > e.g. CallQueueTooBigException. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24086) Disable output stream capability enforcement when running in standalone mode
[
https://issues.apache.org/jira/browse/HBASE-24086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-24086:
--
Component/s: Operability
Priority: Critical (was: Major)
> Disable output stream capability enforcement when running in standalone mode
>
>
> Key: HBASE-24086
> URL: https://issues.apache.org/jira/browse/HBASE-24086
> Project: HBase
> Issue Type: Task
> Components: master, Operability
>Affects Versions: 3.0.0-alpha-1, 2.3.0
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 2.3.0, 1.7.0, 2.2.5
>
>
> {noformat}
> $
> JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home
> mvn clean install -DskipTests
> $
> JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home
> ./bin/hbase master start
> {noformat}
> gives
> {noformat}
> 2020-03-30 17:12:43,857 ERROR
> [master/192.168.111.13:16000:becomeActiveMaster] master.HMaster: Failed to
> become active master
>
> java.io.IOException: cannot get log writer
>
>
> at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:118)
>
>
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:704)
>
>
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:710)
>
>
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128)
>
>
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:839)
>
>
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:549)
>
>
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:490)
>
>
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:156)
>
>
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:61)
>
>
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:297)
>
>
> at
> org.apache.hadoop.hbase.procedure2.store.region.RegionProcedureStore.createWAL(RegionProcedureStore.java:256)
>
>
> at
> org.apache.hadoop.hbase.procedure2.store.region.RegionProcedureStore.bootstrap(RegionProcedureStore.java:273)
>
>
> at
> org.apache.hadoop.hbase.procedure2.store.region.RegionProcedureStore.recoverLease(RegionProcedureStore.java:482)
>
>
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:587)
>
>
> at
> org.apache.hadoop.hbase.master.HMaster.createProcedureExecutor(HMaster.java:1575)
>
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:961)
>
>
[jira] [Updated] (HBASE-20252) Admin.move will not fail if we move region to a nonexistent region server
[ https://issues.apache.org/jira/browse/HBASE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-20252: -- Fix Version/s: (was: 2.3.0) (was: 3.0.0-alpha-1) > Admin.move will not fail if we move region to a nonexistent region server > -- > > Key: HBASE-20252 > URL: https://issues.apache.org/jira/browse/HBASE-20252 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Michael Stack >Priority: Major > Attachments: HBASE-20252-UT.patch > > > The region will just be reopened on the source regionserver... > This is a bit confusing I think... -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-20252) Admin.move will not fail if we move region to a nonexistent region server
[ https://issues.apache.org/jira/browse/HBASE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100349#comment-17100349 ] Michael Stack commented on HBASE-20252: --- Unscheduled it for now. > Admin.move will not fail if we move region to a nonexistent region server > -- > > Key: HBASE-20252 > URL: https://issues.apache.org/jira/browse/HBASE-20252 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Michael Stack >Priority: Major > Attachments: HBASE-20252-UT.patch > > > The region will just be reopened on the source regionserver... > This is a bit confusing I think... -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1656: HBASE-24307 [Flakey Tests] krb server for secure thrift tests throws …
Apache-HBase commented on pull request #1656: URL: https://github.com/apache/hbase/pull/1656#issuecomment-624373834 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 58s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 18s | branch-2 passed | | +1 :green_heart: | compile | 2m 30s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 50s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 15s | hbase-common in branch-2 failed. | | -0 :warning: | javadoc | 0m 16s | hbase-http in branch-2 failed. | | -0 :warning: | javadoc | 0m 40s | hbase-server in branch-2 failed. | | -0 :warning: | javadoc | 0m 58s | hbase-thrift in branch-2 failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 22s | the patch passed | | +1 :green_heart: | compile | 2m 30s | the patch passed | | +1 :green_heart: | javac | 2m 30s | the patch passed | | +1 :green_heart: | shadedjars | 5m 43s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 15s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 16s | hbase-http in the patch failed. | | -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. | | -0 :warning: | javadoc | 0m 57s | hbase-thrift in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 2s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 0m 44s | hbase-http in the patch passed. | | +1 :green_heart: | unit | 179m 59s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 4m 47s | hbase-thrift in the patch passed. | | | | 226m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1656 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 99766c66e02e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 5dc10dbba5 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-http.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-http.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-thrift.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/2/testReport/ | | Max. process+thread count | 2825 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-http hbase-server hbase-thrift U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1656/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git
[jira] [Commented] (HBASE-20252) Admin.move will not fail if we move region to a nonexistent region server
[
https://issues.apache.org/jira/browse/HBASE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100346#comment-17100346
]
Michael Stack commented on HBASE-20252:
---
Just tried this. Get:
{code}
2020-05-06 00:05:45,620 DEBUG org.apache.hadoop.hbase.master.ServerManager: New
admin connection to hbasexy001.example.org,16020,1584993667653
2020-05-06 00:05:45,624 WARN org.apache.hadoop.hbase.client.ConnectionUtils:
Can not resolve hbaseXY001.example.org, please check your network
java.net.UnknownHostException: hbaseXY001.example.org: Name or service not known
at java.base/java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at
java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929)
at
java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1515)
at
java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1364)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
at java.base/java.net.InetAddress.getByName(InetAddress.java:1248)
at
org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:242)
at
org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1224)
at
org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:157)
at
org.apache.hadoop.hbase.master.ServerManager.getRsAdmin(ServerManager.java:723)
at
org.apache.hadoop.hbase.master.ServerManager.sendRegionWarmup(ServerManager.java:662)
at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:2077)
at
org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1437)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:388)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
{code}
Let me mess w/ the Duo UT.
> Admin.move will not fail if we move region to a nonexistent region server
> --
>
> Key: HBASE-20252
> URL: https://issues.apache.org/jira/browse/HBASE-20252
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
> Attachments: HBASE-20252-UT.patch
>
>
> The region will just be reopened on the source regionserver...
> This is a bit confusing I think...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] huaxiangsun opened a new pull request #1662: Backport: HBASE-24273 HBCK's "Orphan Regions on FileSystem" reports regions wit…
huaxiangsun opened a new pull request #1662: URL: https://github.com/apache/hbase/pull/1662 …h referenced HFiles (#1613) Signed-off-by: stack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1660: HBASE-24316 GCMulitpleMergedRegionsProcedure is not idempotent
Apache-HBase commented on pull request #1660: URL: https://github.com/apache/hbase/pull/1660#issuecomment-624371668 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 8m 47s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 32s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 4s | master passed | | +1 :green_heart: | checkstyle | 1m 42s | master passed | | +1 :green_heart: | spotbugs | 3m 16s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 56s | the patch passed | | -0 :warning: | checkstyle | 1m 11s | hbase-server: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 12s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 28s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. | | | | 48m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1660/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1660 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 38272df7f81e 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3d96007c37 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1660/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1660/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun opened a new pull request #1661: Backport: HBASE-24273 HBCK's "Orphan Regions on FileSystem" reports regions wit…
huaxiangsun opened a new pull request #1661: URL: https://github.com/apache/hbase/pull/1661 …h referenced HFiles (#1613) Signed-off-by: stack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bharathv commented on pull request #1593: HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry …
bharathv commented on pull request #1593: URL: https://github.com/apache/hbase/pull/1593#issuecomment-624369936 Looks like the test class is missing a class rule (precommit failure). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bharathv commented on a change in pull request #1593: HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry …
bharathv commented on a change in pull request #1593:
URL: https://github.com/apache/hbase/pull/1593#discussion_r420469468
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterRegistry.java
##
@@ -61,53 +65,79 @@
/**
* Master based registry implementation. Makes RPCs to the configured master
addresses from config
* {@value org.apache.hadoop.hbase.HConstants#MASTER_ADDRS_KEY}.
- *
+ *
* It supports hedged reads, which can be enabled by setting
* {@value
org.apache.hadoop.hbase.HConstants#MASTER_REGISTRY_ENABLE_HEDGED_READS_KEY} to
True. Fan
* out the requests batch is controlled by
* {@value
org.apache.hadoop.hbase.HConstants#HBASE_RPCS_HEDGED_REQS_FANOUT_KEY}.
- *
+ *
* TODO: Handle changes to the configuration dynamically without having to
restart the client.
*/
@InterfaceAudience.Private
public class MasterRegistry implements ConnectionRegistry {
+
+ /** Configuration key that controls the fan out of requests **/
+ public static final String MASTER_REGISTRY_HEDGED_REQS_FANOUT_KEY =
+"hbase.client.master_registry.hedged.fanout";
+
+ /** Default value for the fan out of hedged requests. **/
+ public static final int MASTER_REGISTRY_HEDGED_REQS_FANOUT_DEFAULT = 2;
+
private static final String MASTER_ADDRS_CONF_SEPARATOR = ",";
+ private final int hedgedReadFanOut;
+
// Configured list of masters to probe the meta information from.
- private final Set masterServers;
+ private final Set masterAddrs;
+
+ private final List masterStubs;
// RPC client used to talk to the masters.
private final RpcClient rpcClient;
private final RpcControllerFactory rpcControllerFactory;
- private final int rpcTimeoutMs;
-
- MasterRegistry(Configuration conf) throws UnknownHostException {
-boolean hedgedReadsEnabled =
conf.getBoolean(MASTER_REGISTRY_ENABLE_HEDGED_READS_KEY,
-MASTER_REGISTRY_ENABLE_HEDGED_READS_DEFAULT);
-Configuration finalConf;
-if (!hedgedReadsEnabled) {
- // If hedged reads are disabled, it is equivalent to setting a fan out
of 1. We make a copy of
- // the configuration so that other places reusing this reference is not
affected.
- finalConf = new Configuration(conf);
- finalConf.setInt(HConstants.HBASE_RPCS_HEDGED_REQS_FANOUT_KEY, 1);
-} else {
- finalConf = conf;
+
+ /**
+ * Parses the list of master addresses from the provided configuration.
Supported format is comma
+ * separated host[:port] values. If no port number if specified, default
master port is assumed.
+ * @param conf Configuration to parse from.
+ */
+ private static Set parseMasterAddrs(Configuration conf) throws
UnknownHostException {
+Set masterAddrs = new HashSet<>();
+String configuredMasters = getMasterAddr(conf);
+for (String masterAddr :
configuredMasters.split(MASTER_ADDRS_CONF_SEPARATOR)) {
+ HostAndPort masterHostPort =
+
HostAndPort.fromString(masterAddr.trim()).withDefaultPort(HConstants.DEFAULT_MASTER_PORT);
+ masterAddrs.add(ServerName.valueOf(masterHostPort.toString(),
ServerName.NON_STARTCODE));
}
-if (conf.get(MASTER_ADDRS_KEY) != null) {
- finalConf.set(MASTER_ADDRS_KEY, conf.get(MASTER_ADDRS_KEY));
+Preconditions.checkArgument(!masterAddrs.isEmpty(), "At least one master
address is needed");
+return masterAddrs;
+ }
+
+ MasterRegistry(Configuration conf) throws IOException {
+this.hedgedReadFanOut = conf.getInt(MASTER_REGISTRY_HEDGED_REQS_FANOUT_KEY,
+ MASTER_REGISTRY_HEDGED_REQS_FANOUT_DEFAULT);
+int rpcTimeoutMs = (int) Math.min(Integer.MAX_VALUE,
+ conf.getLong(HConstants.HBASE_RPC_TIMEOUT_KEY,
HConstants.DEFAULT_HBASE_RPC_TIMEOUT));
+// XXX: we pass cluster id as null here since we do not have a cluster id
yet, we have to fetch
Review comment:
Ya, thats what I meant.
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterRegistry.java
##
@@ -117,105 +147,97 @@ public static String getMasterAddr(Configuration conf)
throws UnknownHostExcepti
return String.format("%s:%d", hostname, port);
}
- /**
- * @return Stub needed to make RPC using a hedged channel to the master end
points.
- */
- private ClientMetaService.Interface getMasterStub() throws IOException {
-return ClientMetaService.newStub(
-rpcClient.createHedgedRpcChannel(masterServers, User.getCurrent(),
rpcTimeoutMs));
+ @FunctionalInterface
+ private interface Callable {
+void call(HBaseRpcController controller, ClientMetaService.Interface stub,
RpcCallback done);
}
- /**
- * Parses the list of master addresses from the provided configuration.
Supported format is
- * comma separated host[:port] values. If no port number if specified,
default master port is
- * assumed.
- * @param conf Configuration to parse from.
- */
- private void parseMasterAddrs(Configuration conf) throws
UnknownHostException {
-String con
[jira] [Commented] (HBASE-24273) HBCK's "Orphan Regions on FileSystem" reports regions with referenced HFiles
[ https://issues.apache.org/jira/browse/HBASE-24273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100337#comment-17100337 ] Huaxiang Sun commented on HBASE-24273: -- I am still backport the patch from master to branch-2 and 2.3, wait one second, will update once they are merged. > HBCK's "Orphan Regions on FileSystem" reports regions with referenced HFiles > > > Key: HBASE-24273 > URL: https://issues.apache.org/jira/browse/HBASE-24273 > Project: HBase > Issue Type: Bug > Components: hbck2 >Affects Versions: 2.2.4 > Environment: HBase 2.2.4 > Hadoop 3.1.3 >Reporter: Andrey Elenskiy >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.3.0 > > > This issue came up after merging regions. MergeTableRegionsProcedure removes > the parent regions from hbase:meta and creates HFile references in child > region to the old parent regions. Running `hbck_chore_run` right after the > `merge_region` will show the parent regions in "Orphan Regions on FileSystem" > until major compaction is run on child region which will remove HFile > references and cause Catalog Janitor to clean up the parent regions. > There are probably other situations which can cause the same issue (maybe > region split?) > Having "Orphan Regions on FileSystem" list parent regions and suggest to > "_hbase completebulkload_" is dangerous in this case as completing bulk load > will lead to stale HFile references in child region which will cause its OPEN > to fail because referenced HFile doesn't exist. > Figuring out these things for database administrators is tedious, so I think > it would be reasonable to not consider regions with referenced HFiles to be > orphans (or maybe could give an extra hint saying that it has referenced > HFiles). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23785) Update docs re: Hadoop compatibility of 2.3.x release
[ https://issues.apache.org/jira/browse/HBASE-23785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100336#comment-17100336 ] Michael Stack commented on HBASE-23785: --- Needs note on new zk version and upgrade issues if coming from old version? > Update docs re: Hadoop compatibility of 2.3.x release > - > > Key: HBASE-23785 > URL: https://issues.apache.org/jira/browse/HBASE-23785 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Nick Dimiduk >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0 > > > Update the [book|http://hbase.apache.org/book.html#hadoop] regarding our > Hadoop compatibility for the 2.3 release lines. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24273) HBCK's "Orphan Regions on FileSystem" reports regions with referenced HFiles
[ https://issues.apache.org/jira/browse/HBASE-24273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100334#comment-17100334 ] Andrey Elenskiy commented on HBASE-24273: - Great, thanks for a quick fix! > HBCK's "Orphan Regions on FileSystem" reports regions with referenced HFiles > > > Key: HBASE-24273 > URL: https://issues.apache.org/jira/browse/HBASE-24273 > Project: HBase > Issue Type: Bug > Components: hbck2 >Affects Versions: 2.2.4 > Environment: HBase 2.2.4 > Hadoop 3.1.3 >Reporter: Andrey Elenskiy >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.3.0 > > > This issue came up after merging regions. MergeTableRegionsProcedure removes > the parent regions from hbase:meta and creates HFile references in child > region to the old parent regions. Running `hbck_chore_run` right after the > `merge_region` will show the parent regions in "Orphan Regions on FileSystem" > until major compaction is run on child region which will remove HFile > references and cause Catalog Janitor to clean up the parent regions. > There are probably other situations which can cause the same issue (maybe > region split?) > Having "Orphan Regions on FileSystem" list parent regions and suggest to > "_hbase completebulkload_" is dangerous in this case as completing bulk load > will lead to stale HFile references in child region which will cause its OPEN > to fail because referenced HFile doesn't exist. > Figuring out these things for database administrators is tedious, so I think > it would be reasonable to not consider regions with referenced HFiles to be > orphans (or maybe could give an extra hint saying that it has referenced > HFiles). -- This message was sent by Atlassian Jira (v8.3.4#803005)
