IIRC I copied the docker release originally from HBase, which took them
from Spark.
The M1 issues may have been already fixed in one of those projects.
A simple Ubuntu base image upgrade to 22.04 may fix the M1 specific issues.
I can't help directly, as I do not have access to a Mac, but ping me on
Slack if you get stuck.
As for the third issue, the scripts generate logs in the working directory.
If they do not log the maven command line, you could easily add a line to
log them.
The ERRORS logged are a known issue, as Maven does not like the tricks used
for multi-profile building, but even 3.9.6 accepts them, and only logs
WARNINGs in my experience.
I'm going to do a dry-run of the release scripts locally, and see if I can
repro some problems on my Intel Linux machine.
If you have access to a secure Intel Linux host, you may also want to try
to run the scripts there.
(though getting the ssh password entry working can be tricky)
Istvan
On Sun, Feb 25, 2024 at 9:37 PM Viraj Jasani <vjas...@apache.org> wrote:
> Hi,
>
> I have started with creating 5.2.0 RC, I am starting this thread to discuss
> some of the issues I have come across so far.
>
> 1) do-release-docker.sh is not able to grep and identify snapshot and
> release versions in release-utils.
> While the function parse_version works fine, if run manually on the 5.2 pom
> contents. Hence, I manually updated the utility to take 5.2.0-SNAPSHOT
> version:
>
> --- a/dev/create-release/release-util.sh
> +++ b/dev/create-release/release-util.sh
> @@ -149,6 +149,7 @@ function get_release_info {
> local version
> version="$(curl -s
> "$ASF_REPO_WEBUI;a=blob_plain;f=pom.xml;hb=refs/heads/$GIT_BRANCH" |
> parse_version)"
> + version="5.2.0-SNAPSHOT"
> echo "Current branch VERSION is $version."
>
> RELEASE_VERSION=""
>
>
> This is done to unblock the release for now. We can investigate and fix
> this later.
>
> 2) openjdk-8-amd64 installation fails because I am using M1 Mac:
>
> Setting up openjdk-8-jdk:arm64 (8u372-ga~us1-0ubuntu1~18.04) ...
> update-alternatives: using
> /usr/lib/jvm/java-8-openjdk-arm64/bin/appletviewer to provide
> /usr/bin/appletviewer (appletviewer) in auto mode
> update-alternatives: using /usr/lib/jvm/java-8-openjdk-arm64/bin/jconsole
> to provide /usr/bin/jconsole (jconsole) in auto mode
> Setting up ubuntu-mono (16.10+18.04.20181005-0ubuntu1) ...
> Processing triggers for libc-bin (2.27-3ubuntu1.6) ...
> Processing triggers for ca-certificates (20230311ubuntu0.18.04.1) ...
> Updating certificates in /etc/ssl/certs...
> 0 added, 0 removed; done.
> Running hooks in /etc/ca-certificates/update.d...
> done.
> done.
> Processing triggers for libgdk-pixbuf2.0-0:arm64 (2.36.11-2) ...
> update-alternatives: error: alternative
> /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java for java not registered; not
> setting
>
> In order to resolve this, I set java to use java-8-openjdk-arm64 instead.
> e.g. update-alternatives --set java
> /usr/lib/jvm/java-8-openjdk-arm64/jre/bin/java
> (and all other places where we use amd64)
>
> This is done to make the release progress, we can fix this later.
>
> 3) make_binary_release fails as it is unable to resolve ${hbase.version}
> and ${hbase.compat.version}
>
> Packaging release source tarballs
> 2024-02-25T19:43:46Z make_src_release start
> 2024-02-25T19:43:47Z make_src_release stop (1 seconds)
> 2024-02-25T19:43:47Z make_binary_release start
> 19:45:27 [INFO] Scanning for projects...
> 19:45:27 [ERROR] [ERROR] Some problems were encountered while processing
> the POMs:
> [ERROR] 'dependencies.dependency.artifactId' for
> org.apache.phoenix:phoenix-hbase-compat-${hbase.compat.version}:jar with
> value 'phoenix-hbase-compat-${hbase.compat.version}' does not match a valid
> id pattern. @ org.apache.phoenix:phoenix-core-client:[unknown-version],
> /home/vjasani/phoenix-rm/output/phoenix/phoenix-core-client/pom.xml, line
> 220, column 19
> [ERROR] 'dependencies.dependency.version' for
> org.apache.phoenix:phoenix-hbase-compat-${hbase.compat.version}:jar is
> missing. @ org.apache.phoenix:phoenix-core-client:[unknown-version],
> /home/vjasani/phoenix-rm/output/phoenix/phoenix-core-client/pom.xml, line
> 218, column 17
> [ERROR] 'dependencies.dependency.version' for
> org.apache.hbase:hbase-common:jar must be a valid version but is
> '${hbase.version}'. @ org.apache.phoenix:phoenix:5.2.0,
> /home/vjasani/phoenix-rm/output/phoenix/pom.xml, line 1128, column 18
> [ERROR] 'dependencies.dependency.version' for
> org.apache.hbase:hbase-metrics-api:jar must be a valid version but is
> '${hbase.version}'. @ org.apache.phoenix:phoenix:5.2.0,
> /home/vjasani/phoenix-rm/output/phoenix/pom.xml, line 1151, column 18
> [ERROR] 'dependencies.dependency.version' for
> org.apache.hbase:hbase-client:jar must be a valid version but is
> '${hbase.version}'. @ org.apache.phoenix:phoenix:5.2.0,
> /home/vjasani/phoenix-rm/output/phoenix/pom.xml, line 1161, column 18
> [ERROR] 'dependencies.dependency.version' for
> org.apache.hbase:hbase-hadoop-compat:jar must be a valid version but is
> '${hbase.version}'. @ org.apache.phoenix:phoenix:5.2.0,
> /home/vjasani/phoenix-rm/output/phoenix/pom.xml, line 1226, column 18
> ...
> ...
>
>
> As I do not see "Hbase version is already compiled for Hadoop3. Skipping
> rebuild", I assume this should be the first profile from the profile.list
> i.e. 2.4 and we are unable to build for the first profile.
>
> While 1) and 2) have workarounds, 3) is currently blocking the release.
>
--
*István Tóth* | Sr. Staff Software Engineer
*Email*: st...@cloudera.com
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
------------------------------
