[jira] [Created] (HBASE-24617) Enable injecting build start time value as part of build
Matthew Foley created HBASE-24617:
-
Summary: Enable injecting build start time value as part of build
Key: HBASE-24617
URL: https://issues.apache.org/jira/browse/HBASE-24617
Project: HBase
Issue Type: Improvement
Components: create-release, UI
Affects Versions: 3.0.0-alpha-1, 2.3.0
Reporter: Matthew Foley
Assignee: Matthew Foley
The HBase build's creation time is presented in the HBase UI, and made
available through Java, via the {{org.apache.hadoop.hbase.Version}} class's
{{date}} value, which is generated at build time by
{{hbase-common/src/saveVersion.sh}}. The script just invokes the shell command
{{date}} and captures its result as a string.
The problem is, this occurs every time hbase-common is built. And, for good and
sufficient reason, when making a release via dev-support/create-release, the
task for building and deploying hbase jars as maven libraries and the task for
building binary release artifacts as tarballs, EACH do a {{clean}} build. Thus,
the build time found in the libs is different from the build time found in the
release tarballs.
There is value in keeping the two tasks independent, and able to run fully each
by themselves. And there is value in doing a {{clean}} at the start of such
processes, to make sure you're releasing binaries that exactly match the source
code.
So to keep these benefits, but enable the start time to be determined once and
used for a couple builds in a row in a given environment, I propose to allow
injecting the desired value. Specifically, I want to change saveVersion.sh to
look for an existing value of env var HBASE_BUILD_TIME, and if it exists use it
instead of calling {{date}}. One would of course set it as part of the build
process (in create-release) and clear this value by unsetting the environment
variable when done with the build.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24589) In HBase UI, srcChecksum has bad content
[ https://issues.apache.org/jira/browse/HBASE-24589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Foley updated HBASE-24589: -- Description: The value of srcChecksum displayed in the HBase UI is generated at build time by https://github.com/apache/hbase/blob/master/hbase-common/src/saveVersion.sh#L53-L61 On a Linux build host, if `openssl`(v 1.1.1) is available, the resulting value of srcChecksum is `(stdin)=` else if `gpg` (v 2.2.19) is available, the value is only the first 8-character substring of the sha512 hash. On a Mac build host, if `openssl`(LibreSSL 2.8.3) is available, the resulting value of srcChecksum is correct else if `gpg` (v 2.2.19) is available, the value is again the first 8-character substring of the sha512 hash. Furthermore, while "sha512" is well defined and gives the same value from all four programs, the associated leader text for file names has a different format, and gpg generates uppercase hex divided into 8-character blocks while openssl generates lowercase hex in a single string. Either of these differences will cause the sum-of-sums generated by openssl and gpg to be different! Which is unacceptable if srcChecksum is to be used for validation. was: The value of srcChecksum displayed in the HBase UI is generated at build time by https://github.com/apache/hbase/blob/master/hbase-common/src/saveVersion.sh#L53-L61 On a Linux build host, if `openssl`(v 1.1.1) is available, the resulting value of srcChecksum is `(stdin)=` else if `gpg` (v 2.2.19) is available, the value is only the first 8-character substring of the sha512 hash. On a Mac build host, if `openssl`(LibreSSL 2.8.3) is available, the resulting value of srcChecksum is correct else if `gpg` (v 2.2.19) is available, the value is again the first 8-character substring of the sha512 hash. Furthermore, while "sha512" is well defined and gives the same value from all four programs, the associated leader text for file names has a different format, therefore the sum-of-sums generated by openssl and gpg will be different! Which is unacceptable if srcChecksum is to be used for validation. > In HBase UI, srcChecksum has bad content > > > Key: HBASE-24589 > URL: https://issues.apache.org/jira/browse/HBASE-24589 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 3.0.0-alpha-1, 2.3.0 >Reporter: Matthew Foley >Assignee: Matthew Foley >Priority: Minor > > The value of srcChecksum displayed in the HBase UI is generated at build time > by > https://github.com/apache/hbase/blob/master/hbase-common/src/saveVersion.sh#L53-L61 > On a Linux build host, > if `openssl`(v 1.1.1) is available, the resulting value of srcChecksum is > `(stdin)=` > else if `gpg` (v 2.2.19) is available, the value is only the first > 8-character substring of the sha512 hash. > On a Mac build host, > if `openssl`(LibreSSL 2.8.3) is available, the resulting value of srcChecksum > is correct > else if `gpg` (v 2.2.19) is available, the value is again the first > 8-character substring of the sha512 hash. > Furthermore, while "sha512" is well defined and gives the same value from all > four programs, the associated leader text for file names has a different > format, and gpg generates uppercase hex divided into 8-character blocks while > openssl generates lowercase hex in a single string. Either of these > differences will cause the sum-of-sums generated by openssl and gpg to be > different! Which is unacceptable if srcChecksum is to be used for validation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24589) In HBase UI, srcChecksum has bad content
[ https://issues.apache.org/jira/browse/HBASE-24589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Foley updated HBASE-24589: -- Description: The value of srcChecksum displayed in the HBase UI is generated at build time by https://github.com/apache/hbase/blob/master/hbase-common/src/saveVersion.sh#L53-L61 On a Linux build host, if `openssl`(v 1.1.1) is available, the resulting value of srcChecksum is `(stdin)=` else if `gpg` (v 2.2.19) is available, the value is only the first 8-character substring of the sha512 hash. On a Mac build host, if `openssl`(LibreSSL 2.8.3) is available, the resulting value of srcChecksum is correct else if `gpg` (v 2.2.19) is available, the value is again the first 8-character substring of the sha512 hash. Furthermore, while "sha512" is well defined and gives the same value from all four programs, the associated leader text for file names has a different format, therefore the sum-of-sums generated by openssl and gpg will be different! Which is unacceptable if srcChecksum is to be used for validation. was: The value of srcChecksum displayed in the HBase UI is generated at build time by https://github.com/apache/hbase/blob/master/hbase-common/src/saveVersion.sh#L53-L61 On a Linux build host, if `openssl`(v 1.1.1) is available, the resulting value of srcChecksum is `(stdin)=` else if `gpg` (v 2.2.19) is available, the value is only the first 8-character substring of the sha512 hash. On a Mac build host, if `openssl`(LibreSSL 2.8.3) is available, the resulting value of srcChecksum is correct else if `gpg` (v 2.2.19) is available, the value is again the first 8-character substring of the sha512 hash. > In HBase UI, srcChecksum has bad content > > > Key: HBASE-24589 > URL: https://issues.apache.org/jira/browse/HBASE-24589 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 3.0.0-alpha-1, 2.3.0 >Reporter: Matthew Foley >Assignee: Matthew Foley >Priority: Minor > > The value of srcChecksum displayed in the HBase UI is generated at build time > by > https://github.com/apache/hbase/blob/master/hbase-common/src/saveVersion.sh#L53-L61 > On a Linux build host, > if `openssl`(v 1.1.1) is available, the resulting value of srcChecksum is > `(stdin)=` > else if `gpg` (v 2.2.19) is available, the value is only the first > 8-character substring of the sha512 hash. > On a Mac build host, > if `openssl`(LibreSSL 2.8.3) is available, the resulting value of srcChecksum > is correct > else if `gpg` (v 2.2.19) is available, the value is again the first > 8-character substring of the sha512 hash. > Furthermore, while "sha512" is well defined and gives the same value from all > four programs, the associated leader text for file names has a different > format, therefore the sum-of-sums generated by openssl and gpg will be > different! Which is unacceptable if srcChecksum is to be used for validation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24589) In HBase UI, srcChecksum has bad content
Matthew Foley created HBASE-24589: - Summary: In HBase UI, srcChecksum has bad content Key: HBASE-24589 URL: https://issues.apache.org/jira/browse/HBASE-24589 Project: HBase Issue Type: Bug Components: UI Affects Versions: 3.0.0-alpha-1, 2.3.0 Reporter: Matthew Foley Assignee: Matthew Foley The value of srcChecksum displayed in the HBase UI is generated at build time by https://github.com/apache/hbase/blob/master/hbase-common/src/saveVersion.sh#L53-L61 On a Linux build host, if `openssl`(v 1.1.1) is available, the resulting value of srcChecksum is `(stdin)=` else if `gpg` (v 2.2.19) is available, the value is only the first 8-character substring of the sha512 hash. On a Mac build host, if `openssl`(LibreSSL 2.8.3) is available, the resulting value of srcChecksum is correct else if `gpg` (v 2.2.19) is available, the value is again the first 8-character substring of the sha512 hash. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-24482) [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to missing dependencies
[
https://issues.apache.org/jira/browse/HBASE-24482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17127022#comment-17127022
]
Matthew Foley edited comment on HBASE-24482 at 6/5/20, 7:13 PM:
As an experiment, I changed {{edu.umd.cs.findbugs.annotations.Nullable}} to
{{javax.annotation.Nullable}}, in HBCKMetaTableAccessor.java, and added a
javax.annotation dependency to the hbase-hbck2/pom.xml, as
{{com.google.code.findbugs:jsr305}}
As expected, the original “cannot find symbol … Nullable” on
{{HBCKMetaTableAccessor}} went away, BUT a whole slew of new missing
symbols/dependencies showed up on {{hbck1}} paths. The compile failure log is
attached as
[HBASE-24482_missing_symbols_part_2.log|https://jira.apache.org/jira/secure/attachment/13004958/HBASE-24482_missing_symbols_part_2.log]
was (Author: mattf):
As an experiment, I changed {{edu.umd.cs.findbugs.annotations.Nullable}} to
{{javax.annotation.Nullable}}, and added a javax.annotation dependency to the
hbase-hbck2 pom, as {{com.google.code.findbugs:jsr305}}
As expected, the original “cannot find symbol … Nullable” on
{{HBCKMetaTableAccessor}} went away, BUT a whole slew of new missing
symbols/dependencies showed up on {{hbck1}} paths. The compile failure log is
attached as
[HBASE-24482_missing_symbols_part_2.log|https://jira.apache.org/jira/secure/attachment/13004958/HBASE-24482_missing_symbols_part_2.log]
> [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to
> missing dependencies
> --
>
> Key: HBASE-24482
> URL: https://issues.apache.org/jira/browse/HBASE-24482
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.1.0
>Reporter: Matthew Foley
>Priority: Major
> Attachments: HBASE-24482_compile_error.log,
> HBASE-24482_missing_symbols_part_2.log
>
>
> If we do a local build of the current HBase branch-2.3 as version
> 2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
> then attempt to build hbase-operator-tools master branch, with
> -Dhbase.version=2.3.0-SNAPSHOT,
> the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile
> with error: "package edu.umd.cs.findbugs.annotations does not exist",
> followed by multiple instances of missing "symbol: class Nullable". (A
> longer log extract is attached.)
> It would appear the default HBase version 2.1.6 made this dependency package
> available in some manner, but the newer HBase does not.
> *Later Edit:* After addressing this missing dependency (see comment of 5 June
> below), it unmasked a bunch more, in the hbck1 code path. Please see second
> attachment.
> If the @Nullable annotation is still needed, it may be sufficient to simply
> add it to the hbase-operator-tools pom.xml file, so maven will load it.
> However, it is actually deprecated in the current documentation for
> edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
> should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see
> eg
> *
> [https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
> Does anyone know if that was HBase's approach?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HBASE-24482) [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to missing dependencies
[
https://issues.apache.org/jira/browse/HBASE-24482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17127022#comment-17127022
]
Matthew Foley edited comment on HBASE-24482 at 6/5/20, 7:11 PM:
As an experiment, I changed {{edu.umd.cs.findbugs.annotations.Nullable}} to
{{javax.annotation.Nullable}}, and added a javax.annotation dependency to the
hbase-hbck2 pom, as {{com.google.code.findbugs:jsr305}}
As expected, the original “cannot find symbol … Nullable” on
{{HBCKMetaTableAccessor}} went away, BUT a whole slew of new missing
symbols/dependencies showed up on {{hbck1}} paths. The compile failure log is
attached as
[HBASE-24482_missing_symbols_part_2.log|https://jira.apache.org/jira/secure/attachment/13004958/HBASE-24482_missing_symbols_part_2.log]
was (Author: mattf):
As an experiment, I changed {{edu.umd.cs.findbugs.annotations.Nullable}} to
{{javax.annotation.Nullable}}, and added a javax.annotation dependency to the
hbase-hbck2 pom, as {{com.google.code.findbugs:jsr305}}
As expected, the original “cannot find symbol … Nullable” on
{{HBCKMetaTableAccessor}} went away, BUT a whole slew of new missing
symbols/dependencies showed up on {{hbck1}} paths. The compile failure log is
attached as [#HBASE-24482_missing_symbols_part_2.log]
> [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to
> missing dependencies
> --
>
> Key: HBASE-24482
> URL: https://issues.apache.org/jira/browse/HBASE-24482
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.1.0
>Reporter: Matthew Foley
>Priority: Major
> Attachments: HBASE-24482_compile_error.log,
> HBASE-24482_missing_symbols_part_2.log
>
>
> If we do a local build of the current HBase branch-2.3 as version
> 2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
> then attempt to build hbase-operator-tools master branch, with
> -Dhbase.version=2.3.0-SNAPSHOT,
> the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile
> with error: "package edu.umd.cs.findbugs.annotations does not exist",
> followed by multiple instances of missing "symbol: class Nullable". (A
> longer log extract is attached.)
> It would appear the default HBase version 2.1.6 made this dependency package
> available in some manner, but the newer HBase does not.
> *Later Edit:* After addressing this missing dependency (see comment of 5 June
> below), it unmasked a bunch more, in the hbck1 code path. Please see second
> attachment.
> If the @Nullable annotation is still needed, it may be sufficient to simply
> add it to the hbase-operator-tools pom.xml file, so maven will load it.
> However, it is actually deprecated in the current documentation for
> edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
> should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see
> eg
> *
> [https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
> Does anyone know if that was HBase's approach?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HBASE-24482) [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to missing dependencies
[
https://issues.apache.org/jira/browse/HBASE-24482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17127022#comment-17127022
]
Matthew Foley edited comment on HBASE-24482 at 6/5/20, 7:10 PM:
As an experiment, I changed {{edu.umd.cs.findbugs.annotations.Nullable}} to
{{javax.annotation.Nullable}}, and added a javax.annotation dependency to the
hbase-hbck2 pom, as {{com.google.code.findbugs:jsr305}}
As expected, the original “cannot find symbol … Nullable” on
{{HBCKMetaTableAccessor}} went away, BUT a whole slew of new missing
symbols/dependencies showed up on {{hbck1}} paths. The compile failure log is
attached as [#HBASE-24482_missing_symbols_part_2.log]
was (Author: mattf):
As an experiment, I changed {{edu.umd.cs.findbugs.annotations.Nullable}} to
{{javax.annotation.Nullable}}, and added a javax.annotation dependency to the
hbase-hbck2 pom, as {{com.google.code.findbugs:jsr305}}
{{}}As expected, the original “cannot find symbol … Nullable” on
{{HBCKMetaTableAccessor}} went away, BUT a whole slew of new missing
symbols/dependencies showed up on {{hbck1}} paths. The compile failure log is
attached as HBASE-24482_missing_symbols_part_2.log
> [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to
> missing dependencies
> --
>
> Key: HBASE-24482
> URL: https://issues.apache.org/jira/browse/HBASE-24482
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.1.0
>Reporter: Matthew Foley
>Priority: Major
> Attachments: HBASE-24482_compile_error.log,
> HBASE-24482_missing_symbols_part_2.log
>
>
> If we do a local build of the current HBase branch-2.3 as version
> 2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
> then attempt to build hbase-operator-tools master branch, with
> -Dhbase.version=2.3.0-SNAPSHOT,
> the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile
> with error: "package edu.umd.cs.findbugs.annotations does not exist",
> followed by multiple instances of missing "symbol: class Nullable". (A
> longer log extract is attached.)
> It would appear the default HBase version 2.1.6 made this dependency package
> available in some manner, but the newer HBase does not.
> *Later Edit:* After addressing this missing dependency (see comment of 5 June
> below), it unmasked a bunch more, in the hbck1 code path. Please see second
> attachment.
> If the @Nullable annotation is still needed, it may be sufficient to simply
> add it to the hbase-operator-tools pom.xml file, so maven will load it.
> However, it is actually deprecated in the current documentation for
> edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
> should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see
> eg
> *
> [https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
> Does anyone know if that was HBase's approach?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24482) [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to missing dependencies
[
https://issues.apache.org/jira/browse/HBASE-24482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17127022#comment-17127022
]
Matthew Foley commented on HBASE-24482:
---
As an experiment, I changed {{edu.umd.cs.findbugs.annotations.Nullable}} to
{{javax.annotation.Nullable}}, and added a javax.annotation dependency to the
hbase-hbck2 pom, as {{com.google.code.findbugs:jsr305}}
{{}}As expected, the original “cannot find symbol … Nullable” on
{{HBCKMetaTableAccessor}} went away, BUT a whole slew of new missing
symbols/dependencies showed up on {{hbck1}} paths. The compile failure log is
attached as HBASE-24482_missing_symbols_part_2.log
> [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to
> missing dependencies
> --
>
> Key: HBASE-24482
> URL: https://issues.apache.org/jira/browse/HBASE-24482
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.1.0
>Reporter: Matthew Foley
>Priority: Major
> Attachments: HBASE-24482_compile_error.log,
> HBASE-24482_missing_symbols_part_2.log
>
>
> If we do a local build of the current HBase branch-2.3 as version
> 2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
> then attempt to build hbase-operator-tools master branch, with
> -Dhbase.version=2.3.0-SNAPSHOT,
> the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile
> with error: "package edu.umd.cs.findbugs.annotations does not exist",
> followed by multiple instances of missing "symbol: class Nullable". (A
> longer log extract is attached.)
> It would appear the default HBase version 2.1.6 made this dependency package
> available in some manner, but the newer HBase does not.
> *Later Edit:* After addressing this missing dependency (see comment of 5 June
> below), it unmasked a bunch more, in the hbck1 code path. Please see second
> attachment.
> If the @Nullable annotation is still needed, it may be sufficient to simply
> add it to the hbase-operator-tools pom.xml file, so maven will load it.
> However, it is actually deprecated in the current documentation for
> edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
> should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see
> eg
> *
> [https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
> Does anyone know if that was HBase's approach?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24482) [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to missing dependencies
[
https://issues.apache.org/jira/browse/HBASE-24482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthew Foley updated HBASE-24482:
--
Attachment: HBASE-24482_missing_symbols_part_2.log
> [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to
> missing dependencies
> --
>
> Key: HBASE-24482
> URL: https://issues.apache.org/jira/browse/HBASE-24482
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.1.0
>Reporter: Matthew Foley
>Priority: Major
> Attachments: HBASE-24482_compile_error.log,
> HBASE-24482_missing_symbols_part_2.log
>
>
> If we do a local build of the current HBase branch-2.3 as version
> 2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
> then attempt to build hbase-operator-tools master branch, with
> -Dhbase.version=2.3.0-SNAPSHOT,
> the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile
> with error: "package edu.umd.cs.findbugs.annotations does not exist",
> followed by multiple instances of missing "symbol: class Nullable". (A
> longer log extract is attached.)
> It would appear the default HBase version 2.1.6 made this dependency package
> available in some manner, but the newer HBase does not.
> *Later Edit:* After addressing this missing dependency (see comment of 5 June
> below), it unmasked a bunch more, in the hbck1 code path. Please see second
> attachment.
> If the @Nullable annotation is still needed, it may be sufficient to simply
> add it to the hbase-operator-tools pom.xml file, so maven will load it.
> However, it is actually deprecated in the current documentation for
> edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
> should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see
> eg
> *
> [https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
> Does anyone know if that was HBase's approach?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24482) [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to missing dependencies
[
https://issues.apache.org/jira/browse/HBASE-24482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthew Foley updated HBASE-24482:
--
Description:
If we do a local build of the current HBase branch-2.3 as version
2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
then attempt to build hbase-operator-tools master branch, with
-Dhbase.version=2.3.0-SNAPSHOT,
the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile with
error: "package edu.umd.cs.findbugs.annotations does not exist", followed by
multiple instances of missing "symbol: class Nullable". (A longer log extract
is attached.)
It would appear the default HBase version 2.1.6 made this dependency package
available in some manner, but the newer HBase does not.
*Later Edit:* After addressing this missing dependency (see comment of 5 June
below), it unmasked a bunch more, in the hbck1 code path. Please see second
attachment.
If the @Nullable annotation is still needed, it may be sufficient to simply add
it to the hbase-operator-tools pom.xml file, so maven will load it. However,
it is actually deprecated in the current documentation for
edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see eg
*
[https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
Does anyone know if that was HBase's approach?
was:
If we do a local build of the current HBase branch-2.3 as version
2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
then attempt to build hbase-operator-tools master branch, with
-Dhbase.version=2.3.0-SNAPSHOT,
the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile with
error: "package edu.umd.cs.findbugs.annotations does not exist", followed by
multiple instances of missing "symbol: class Nullable". (A longer log extract
is attached.)
It would appear the default HBase version 2.1.6 made this dependency package
available in some manner, but the newer HBase does not.
If the @Nullable annotation is still needed, it may be sufficient to simply add
it to the hbase-operator-tools pom.xml file, so maven will load it. However,
it is actually deprecated in the current documentation for
edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see eg
*
[https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
Does anyone know if that was HBase's approach?
> [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to
> missing dependencies
> --
>
> Key: HBASE-24482
> URL: https://issues.apache.org/jira/browse/HBASE-24482
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.1.0
>Reporter: Matthew Foley
>Priority: Major
> Attachments: HBASE-24482_compile_error.log
>
>
> If we do a local build of the current HBase branch-2.3 as version
> 2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
> then attempt to build hbase-operator-tools master branch, with
> -Dhbase.version=2.3.0-SNAPSHOT,
> the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile
> with error: "package edu.umd.cs.findbugs.annotations does not exist",
> followed by multiple instances of missing "symbol: class Nullable". (A
> longer log extract is attached.)
> It would appear the default HBase version 2.1.6 made this dependency package
> available in some manner, but the newer HBase does not.
> *Later Edit:* After addressing this missing dependency (see comment of 5 June
> below), it unmasked a bunch more, in the hbck1 code path. Please see second
> attachment.
> If the @Nullable annotation is still needed, it may be sufficient to simply
> add it to the hbase-operator-tools pom.xml file, so maven will load it.
> However, it is actually deprecated in the current documentation for
> edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
> should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see
> eg
> *
> [https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
> Does anyone know if that was HBase's approach?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24482) [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to missing dependencies
[
https://issues.apache.org/jira/browse/HBASE-24482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthew Foley updated HBASE-24482:
--
Summary: [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3,
due to missing dependencies (was: [hbase-operator-tools] build of hbck2 fails
with HBase branch-2.3, due to missing edu.umd.cs.findbugs.annotations
dependency)
> [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to
> missing dependencies
> --
>
> Key: HBASE-24482
> URL: https://issues.apache.org/jira/browse/HBASE-24482
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.1.0
>Reporter: Matthew Foley
>Priority: Major
> Attachments: HBASE-24482_compile_error.log
>
>
> If we do a local build of the current HBase branch-2.3 as version
> 2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
> then attempt to build hbase-operator-tools master branch, with
> -Dhbase.version=2.3.0-SNAPSHOT,
> the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile
> with error: "package edu.umd.cs.findbugs.annotations does not exist",
> followed by multiple instances of missing "symbol: class Nullable". (A
> longer log extract is attached.)
> It would appear the default HBase version 2.1.6 made this dependency package
> available in some manner, but the newer HBase does not.
> If the @Nullable annotation is still needed, it may be sufficient to simply
> add it to the hbase-operator-tools pom.xml file, so maven will load it.
> However, it is actually deprecated in the current documentation for
> edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
> should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see
> eg
> *
> [https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
> Does anyone know if that was HBase's approach?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24482) [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to missing edu.umd.cs.findbugs.annotations dependency
[
https://issues.apache.org/jira/browse/HBASE-24482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120761#comment-17120761
]
Matthew Foley commented on HBASE-24482:
---
Interesting, grepping the branch-2.3 of HBase, I see the poms do _not_ mention
edu.umd.cs.findbugs.annotations, and _do_ mention javax.annotation. BUT there
are still scads of uses of edu.umd.cs.findbugs.annotations in java files. The
tag 2.1.6RC1 has the same behavior. So I infer that the
edu.umd.cs.findbugs.annotations package was being pulled in by an indirect
dependency, and either an assembly or a shadowing somewhere along the line
changed since 2.1.6.
> [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to
> missing edu.umd.cs.findbugs.annotations dependency
>
>
> Key: HBASE-24482
> URL: https://issues.apache.org/jira/browse/HBASE-24482
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.1.0
>Reporter: Matthew Foley
>Priority: Major
> Attachments: HBASE-24482_compile_error.log
>
>
> If we do a local build of the current HBase branch-2.3 as version
> 2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
> then attempt to build hbase-operator-tools master branch, with
> -Dhbase.version=2.3.0-SNAPSHOT,
> the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile
> with error: "package edu.umd.cs.findbugs.annotations does not exist",
> followed by multiple instances of missing "symbol: class Nullable". (A
> longer log extract is attached.)
> It would appear the default HBase version 2.1.6 made this dependency package
> available in some manner, but the newer HBase does not.
> If the @Nullable annotation is still needed, it may be sufficient to simply
> add it to the hbase-operator-tools pom.xml file, so maven will load it.
> However, it is actually deprecated in the current documentation for
> edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
> should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see
> eg
> *
> [https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
> Does anyone know if that was HBase's approach?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24482) [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to missing edu.umd.cs.findbugs.annotations dependency
[
https://issues.apache.org/jira/browse/HBASE-24482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthew Foley updated HBASE-24482:
--
Description:
If we do a local build of the current HBase branch-2.3 as version
2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
then attempt to build hbase-operator-tools master branch, with
-Dhbase.version=2.3.0-SNAPSHOT,
the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile with
error: "package edu.umd.cs.findbugs.annotations does not exist", followed by
multiple instances of missing "symbol: class Nullable". (A longer log extract
is attached.)
It would appear the default HBase version 2.1.6 made this dependency package
available in some manner, but the newer HBase does not.
If the @Nullable annotation is still needed, it may be sufficient to simply add
it to the hbase-operator-tools pom.xml file, so maven will load it. However,
it is actually deprecated in the current documentation for
edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see eg
*
[https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
Does anyone know if that was HBase's approach?
was:
If we do a local build of the current HBase branch-2.3 as version
2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
then attempt to build hbase-operator-tools master branch, with
-Dhbase.version=2.3.0-SNAPSHOT,
the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile with
error: "package edu.umd.cs.findbugs.annotations does not exist", followed by
multiple instances of missing "symbol: class Nullable". (A longer log extract
is attached.)
It would appear the default HBase version 2.1.6 made this dependency package
available in some manner, but the newer HBase does not.
If the @Nullable annotation is still needed, it may be sufficient to simply add
it to the hbase-operator-tools pom.xml file, so maven will load it. However,
it is actually deprecated in the current documentation for
edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see eg
*
[https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
> [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to
> missing edu.umd.cs.findbugs.annotations dependency
>
>
> Key: HBASE-24482
> URL: https://issues.apache.org/jira/browse/HBASE-24482
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.1.0
>Reporter: Matthew Foley
>Priority: Major
> Attachments: HBASE-24482_compile_error.log
>
>
> If we do a local build of the current HBase branch-2.3 as version
> 2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
> then attempt to build hbase-operator-tools master branch, with
> -Dhbase.version=2.3.0-SNAPSHOT,
> the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile
> with error: "package edu.umd.cs.findbugs.annotations does not exist",
> followed by multiple instances of missing "symbol: class Nullable". (A
> longer log extract is attached.)
> It would appear the default HBase version 2.1.6 made this dependency package
> available in some manner, but the newer HBase does not.
> If the @Nullable annotation is still needed, it may be sufficient to simply
> add it to the hbase-operator-tools pom.xml file, so maven will load it.
> However, it is actually deprecated in the current documentation for
> edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
> should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see
> eg
> *
> [https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
> Does anyone know if that was HBase's approach?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24482) [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to missing edu.umd.cs.findbugs.annotations dependency
[
https://issues.apache.org/jira/browse/HBASE-24482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthew Foley updated HBASE-24482:
--
Description:
If we do a local build of the current HBase branch-2.3 as version
2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
then attempt to build hbase-operator-tools master branch, with
-Dhbase.version=2.3.0-SNAPSHOT,
the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile with
error: "package edu.umd.cs.findbugs.annotations does not exist", followed by
multiple instances of missing "symbol: class Nullable". (A longer log extract
is attached.)
It would appear the default HBase version 2.1.6 made this dependency package
available in some manner, but the newer HBase does not.
If the @Nullable annotation is still needed, it may be sufficient to simply add
it to the hbase-operator-tools pom.xml file, so maven will load it. However,
it is actually deprecated in the current documentation for
edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see eg
*
[https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
was:
If we do a local build of the current HBase branch-2.3 as version
2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
then attempt to build hbase-operator-tools master branch, with
-Dhbase.version=2.3.0-SNAPSHOT,
the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile with
error: "package edu.umd.cs.findbugs.annotations does not exist", followed by
multiple instances of missing "symbol: class Nullable". (A longer log extract
is attached.)
It would appear the default HBase version 2.1.6 made this dependency package
available in some manner, but the newer HBase does not.
If the @Nullable annotation is still needed, it may be sufficient to simply add
it to the hbase-operator-tools pom.xml file, so maven will load it. However,
various commentary online suggest that we should be using the similar
jsr305.jar (*{{javax.annotation)}}* instead; see eg
*
[https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
> [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to
> missing edu.umd.cs.findbugs.annotations dependency
>
>
> Key: HBASE-24482
> URL: https://issues.apache.org/jira/browse/HBASE-24482
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.1.0
>Reporter: Matthew Foley
>Priority: Major
> Attachments: HBASE-24482_compile_error.log
>
>
> If we do a local build of the current HBase branch-2.3 as version
> 2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
> then attempt to build hbase-operator-tools master branch, with
> -Dhbase.version=2.3.0-SNAPSHOT,
> the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile
> with error: "package edu.umd.cs.findbugs.annotations does not exist",
> followed by multiple instances of missing "symbol: class Nullable". (A
> longer log extract is attached.)
> It would appear the default HBase version 2.1.6 made this dependency package
> available in some manner, but the newer HBase does not.
> If the @Nullable annotation is still needed, it may be sufficient to simply
> add it to the hbase-operator-tools pom.xml file, so maven will load it.
> However, it is actually deprecated in the current documentation for
> edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
> should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see
> eg
> *
> [https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24482) [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to missing edu.umd.cs.findbugs.annotations dependency
[
https://issues.apache.org/jira/browse/HBASE-24482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthew Foley updated HBASE-24482:
--
Attachment: HBASE-24482_compile_error.log
> [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to
> missing edu.umd.cs.findbugs.annotations dependency
>
>
> Key: HBASE-24482
> URL: https://issues.apache.org/jira/browse/HBASE-24482
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.1.0
>Reporter: Matthew Foley
>Priority: Major
> Attachments: HBASE-24482_compile_error.log
>
>
> If we do a local build of the current HBase branch-2.3 as version
> 2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
> then attempt to build hbase-operator-tools master branch, with
> -Dhbase.version=2.3.0-SNAPSHOT,
> the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile
> with error: "package edu.umd.cs.findbugs.annotations does not exist",
> followed by multiple instances of missing "symbol: class Nullable". (A
> longer log extract is attached.)
> It would appear the default HBase version 2.1.6 made this dependency package
> available in some manner, but the newer HBase does not.
> If the @Nullable annotation is still needed, it may be sufficient to simply
> add it to the hbase-operator-tools pom.xml file, so maven will load it.
> However, various commentary online suggest that we should be using the
> similar jsr305.jar (*{{javax.annotation)}}* instead; see eg
> *
> [https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24482) [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to missing edu.umd.cs.findbugs.annotations dependency
Matthew Foley created HBASE-24482:
-
Summary: [hbase-operator-tools] build of hbck2 fails with HBase
branch-2.3, due to missing edu.umd.cs.findbugs.annotations dependency
Key: HBASE-24482
URL: https://issues.apache.org/jira/browse/HBASE-24482
Project: HBase
Issue Type: Bug
Components: hbase-operator-tools
Affects Versions: hbase-operator-tools-1.1.0
Reporter: Matthew Foley
If we do a local build of the current HBase branch-2.3 as version
2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
then attempt to build hbase-operator-tools master branch, with
-Dhbase.version=2.3.0-SNAPSHOT,
the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile with
error: "package edu.umd.cs.findbugs.annotations does not exist", followed by
multiple instances of missing "symbol: class Nullable". (A longer log extract
is attached.)
It would appear the default HBase version 2.1.6 made this dependency package
available in some manner, but the newer HBase does not.
If the @Nullable annotation is still needed, it may be sufficient to simply add
it to the hbase-operator-tools pom.xml file, so maven will load it. However,
various commentary online suggest that we should be using the similar
jsr305.jar (*{{javax.annotation)}}* instead; see eg
*
[https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-12187) Review in source the paper "Simple Testing Can Prevent Most Critical Failures"
[ https://issues.apache.org/jira/browse/HBASE-12187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113667#comment-17113667 ] Matthew Foley commented on HBASE-12187: --- Not sure whether this ever got done or not, but seeing as the Jira is still open, I thought y'all might be interested to know that a couple days ago, [https://github.com/google/error-prone/pull/372] , consisting of these rules, was finally merged into [error-prone|https://github.com/google/error-prone]! No release yet, (last one was version 2.3.4 on 12/2/2019), but maybe someday :) > Review in source the paper "Simple Testing Can Prevent Most Critical Failures" > -- > > Key: HBASE-12187 > URL: https://issues.apache.org/jira/browse/HBASE-12187 > Project: HBase > Issue Type: Bug >Reporter: Michael Stack >Priority: Critical > Attachments: HBASE-12187.patch, abortInOvercatch.warnings.txt, > emptyCatch.warnings.txt, todoInCatch.warnings.txt > > > Review the helpful paper > https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-yuan.pdf > It describes 'catastrophic failures', especially issues where exceptions are > thrown but not properly handled. Their static analysis tool Aspirator turns > up a bunch of the obvious offenders (Lets add to test-patch.sh alongside > findbugs?). This issue is about going through code base making sub-issues to > root out these and others (Don't we have the test described in figure #6 > already? I thought we did? If we don't, need to add). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24318) Create-release scripts fixes and enhancements
[ https://issues.apache.org/jira/browse/HBASE-24318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Foley resolved HBASE-24318. --- Fix Version/s: 3.0.0-alpha-1 Resolution: Fixed > Create-release scripts fixes and enhancements > - > > Key: HBASE-24318 > URL: https://issues.apache.org/jira/browse/HBASE-24318 > Project: HBase > Issue Type: Improvement > Components: create-release >Affects Versions: 3.0.0-alpha-1, 2.3.0 > Environment: Linux Docker container, or Mac OS X without Docker >Reporter: Matthew Foley >Assignee: Matthew Foley >Priority: Major > Fix For: 3.0.0-alpha-1 > > > The create-release tools are a set of scripts that promote best practice in > making Apache releases, by automating most of the steps and running them > under Docker for reliability. However, the current state of the scripts has > many bugs and ambiguities. > The proposed PR cleans up the code and clarifies usage. It enables: > - Clear statement of the four steps, which are now called `tag`, > `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two > being mutually exclusive alternatives). > - Ability to do the three tag-dist-release steps with a single command, or do > any of the steps singly. (Running singly had bugs and unfulfilled > dependencies before.) > - Ability to do a reliable and useful "dry run" of all steps or each step, > and chain together the tag step with publish steps in a dry run. > - Ability to run any or all steps correctly in Docker or outside of Docker, > on Linux and Mac. > - Cleaned up all `shellcheck` errors in the scripts, and removed ambiguities > and redundancies in the many environment variables used. > In addition, the changes move the code toward being more general / less > HBase-specific, so it can be run on any Apache project (while still > accommodating HBase-specific features regarding how sub-projects are named > and organized in Jira and release repos). In future I propose to take it > further along that path, and move create-release into Yetus (recognizing that > this create-release code has been passed between a couple other projects > already). > These changes have NO IMPACT on HBase functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24318) Create-release scripts fixes and enhancements
[ https://issues.apache.org/jira/browse/HBASE-24318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106519#comment-17106519 ] Matthew Foley commented on HBASE-24318: --- Merged to master as commit f3c0a0e06df80a8a8cc5de0a8c5ab2ccf680f973, by [~busbey] Thanks, Sean! > Create-release scripts fixes and enhancements > - > > Key: HBASE-24318 > URL: https://issues.apache.org/jira/browse/HBASE-24318 > Project: HBase > Issue Type: Improvement > Components: create-release >Affects Versions: 3.0.0-alpha-1, 2.3.0 > Environment: Linux Docker container, or Mac OS X without Docker >Reporter: Matthew Foley >Assignee: Matthew Foley >Priority: Major > > The create-release tools are a set of scripts that promote best practice in > making Apache releases, by automating most of the steps and running them > under Docker for reliability. However, the current state of the scripts has > many bugs and ambiguities. > The proposed PR cleans up the code and clarifies usage. It enables: > - Clear statement of the four steps, which are now called `tag`, > `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two > being mutually exclusive alternatives). > - Ability to do the three tag-dist-release steps with a single command, or do > any of the steps singly. (Running singly had bugs and unfulfilled > dependencies before.) > - Ability to do a reliable and useful "dry run" of all steps or each step, > and chain together the tag step with publish steps in a dry run. > - Ability to run any or all steps correctly in Docker or outside of Docker, > on Linux and Mac. > - Cleaned up all `shellcheck` errors in the scripts, and removed ambiguities > and redundancies in the many environment variables used. > In addition, the changes move the code toward being more general / less > HBase-specific, so it can be run on any Apache project (while still > accommodating HBase-specific features regarding how sub-projects are named > and organized in Jira and release repos). In future I propose to take it > further along that path, and move create-release into Yetus (recognizing that > this create-release code has been passed between a couple other projects > already). > These changes have NO IMPACT on HBase functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24318) Create-release scripts fixes and enhancements
[ https://issues.apache.org/jira/browse/HBASE-24318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Foley updated HBASE-24318: -- Release Note: Release Managers will want to take note of two changes in the create-release tools: - The names of the release steps are now `tag`, `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two being mutually exclusive alternatives). `Publish-dist` was formerly just named `build`, which disguised the fact that it built AND published tarballs to https://dist.apache.org/repos/dist/release/ , and that publish-snapshot/publish-release do their own build before publishing maven artifacts to the Nexus repos. - "Dry run" has become the _default_ behavior. Doing a dry run first decreases the likelihood of failure part-way through, which could result in inconsistent artifacts in the repos. To actually do the release, add the `-f` (force) option flag when you invoke `do-release-docker.sh` or `do-release.sh`. This is similar to `-f` usage in some Linux and git commands that have high cost for mistakes. This is a change from the former `-n` to turn ON dry run. > Create-release scripts fixes and enhancements > - > > Key: HBASE-24318 > URL: https://issues.apache.org/jira/browse/HBASE-24318 > Project: HBase > Issue Type: Improvement > Components: create-release >Affects Versions: 3.0.0-alpha-1, 2.3.0 > Environment: Linux Docker container, or Mac OS X without Docker >Reporter: Matthew Foley >Assignee: Matthew Foley >Priority: Major > > The create-release tools are a set of scripts that promote best practice in > making Apache releases, by automating most of the steps and running them > under Docker for reliability. However, the current state of the scripts has > many bugs and ambiguities. > The proposed PR cleans up the code and clarifies usage. It enables: > - Clear statement of the four steps, which are now called `tag`, > `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two > being mutually exclusive alternatives). > - Ability to do the three tag-dist-release steps with a single command, or do > any of the steps singly. (Running singly had bugs and unfulfilled > dependencies before.) > - Ability to do a reliable and useful "dry run" of all steps or each step, > and chain together the tag step with publish steps in a dry run. > - Ability to run any or all steps correctly in Docker or outside of Docker, > on Linux and Mac. > - Cleaned up all `shellcheck` errors in the scripts, and removed ambiguities > and redundancies in the many environment variables used. > In addition, the changes move the code toward being more general / less > HBase-specific, so it can be run on any Apache project (while still > accommodating HBase-specific features regarding how sub-projects are named > and organized in Jira and release repos). In future I propose to take it > further along that path, and move create-release into Yetus (recognizing that > this create-release code has been passed between a couple other projects > already). > These changes have NO IMPACT on HBase functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HBASE-24318) Create-release scripts fixes and enhancements
[ https://issues.apache.org/jira/browse/HBASE-24318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-24318 started by Matthew Foley. - > Create-release scripts fixes and enhancements > - > > Key: HBASE-24318 > URL: https://issues.apache.org/jira/browse/HBASE-24318 > Project: HBase > Issue Type: Improvement > Components: create-release >Affects Versions: 3.0.0-alpha-1, 2.3.0 > Environment: Linux Docker container, or Mac OS X without Docker >Reporter: Matthew Foley >Assignee: Matthew Foley >Priority: Major > > The create-release tools are a set of scripts that promote best practice in > making Apache releases, by automating most of the steps and running them > under Docker for reliability. However, the current state of the scripts has > many bugs and ambiguities. > The proposed PR cleans up the code and clarifies usage. It enables: > - Clear statement of the four steps, which are now called `tag`, > `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two > being mutually exclusive alternatives). > - Ability to do the three tag-dist-release steps with a single command, or do > any of the steps singly. (Running singly had bugs and unfulfilled > dependencies before.) > - Ability to do a reliable and useful "dry run" of all steps or each step, > and chain together the tag step with publish steps in a dry run. > - Ability to run any or all steps correctly in Docker or outside of Docker, > on Linux and Mac. > - Cleaned up all `shellcheck` errors in the scripts, and removed ambiguities > and redundancies in the many environment variables used. > In addition, the changes move the code toward being more general / less > HBase-specific, so it can be run on any Apache project (while still > accommodating HBase-specific features regarding how sub-projects are named > and organized in Jira and release repos). In future I propose to take it > further along that path, and move create-release into Yetus (recognizing that > this create-release code has been passed between a couple other projects > already). > These changes have NO IMPACT on HBase functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24318) Create-release scripts fixes and enhancements
[ https://issues.apache.org/jira/browse/HBASE-24318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099480#comment-17099480 ] Matthew Foley commented on HBASE-24318: --- Thanks, here's the link. > Create-release scripts fixes and enhancements > - > > Key: HBASE-24318 > URL: https://issues.apache.org/jira/browse/HBASE-24318 > Project: HBase > Issue Type: Improvement > Components: create-release >Affects Versions: 3.0.0-alpha-1, 2.3.0 > Environment: Linux Docker container, or Mac OS X without Docker >Reporter: Matthew Foley >Priority: Major > > The create-release tools are a set of scripts that promote best practice in > making Apache releases, by automating most of the steps and running them > under Docker for reliability. However, the current state of the scripts has > many bugs and ambiguities. > The proposed PR cleans up the code and clarifies usage. It enables: > - Clear statement of the four steps, which are now called `tag`, > `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two > being mutually exclusive alternatives). > - Ability to do the three tag-dist-release steps with a single command, or do > any of the steps singly. (Running singly had bugs and unfulfilled > dependencies before.) > - Ability to do a reliable and useful "dry run" of all steps or each step, > and chain together the tag step with publish steps in a dry run. > - Ability to run any or all steps correctly in Docker or outside of Docker, > on Linux and Mac. > - Cleaned up all `shellcheck` errors in the scripts, and removed ambiguities > and redundancies in the many environment variables used. > In addition, the changes move the code toward being more general / less > HBase-specific, so it can be run on any Apache project (while still > accommodating HBase-specific features regarding how sub-projects are named > and organized in Jira and release repos). In future I propose to take it > further along that path, and move create-release into Yetus (recognizing that > this create-release code has been passed between a couple other projects > already). > These changes have NO IMPACT on HBase functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24318) Create-release scripts fixes and enhancements
Matthew Foley created HBASE-24318: - Summary: Create-release scripts fixes and enhancements Key: HBASE-24318 URL: https://issues.apache.org/jira/browse/HBASE-24318 Project: HBase Issue Type: Improvement Components: create-release Affects Versions: 3.0.0-alpha-1, 2.3.0 Environment: Linux Docker container, or Mac OS X without Docker Reporter: Matthew Foley The create-release tools are a set of scripts that promote best practice in making Apache releases, by automating most of the steps and running them under Docker for reliability. However, the current state of the scripts has many bugs and ambiguities. The proposed PR cleans up the code and clarifies usage. It enables: - Clear statement of the four steps, which are now called `tag`, `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two being mutually exclusive alternatives). - Ability to do the three tag-dist-release steps with a single command, or do any of the steps singly. (Running singly had bugs and unfulfilled dependencies before.) - Ability to do a reliable and useful "dry run" of all steps or each step, and chain together the tag step with publish steps in a dry run. - Ability to run any or all steps correctly in Docker or outside of Docker, on Linux and Mac. - Cleaned up all `shellcheck` errors in the scripts, and removed ambiguities and redundancies in the many environment variables used. In addition, the changes move the code toward being more general / less HBase-specific, so it can be run on any Apache project (while still accommodating HBase-specific features regarding how sub-projects are named and organized in Jira and release repos). In future I propose to take it further along that path, and move create-release into Yetus (recognizing that this create-release code has been passed between a couple other projects already). These changes have NO IMPACT on HBase functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005)
