It seems to be caused by obsolete instruction in HowToRelease Wiki?
After HADOOP-15058, `mvn deploy` is kicked by
`dev-support/bin/create-release --asfrelease`.
https://issues.apache.org/jira/browse/HADOOP-15058
Step #10 in "Creating the release candidate (X.Y.Z-RC<N>)" section
of the Wiki still instructs to run `mvn deploy` with `-DskipShade`.
2 sets of artifact are deployed after creating RC based on the instruction.
The latest one contains empty shaded jars.
hadoop-client-api and hadoop-client-runtime of already released 3.2.3
looks having same issue...
Masatake Iwasaki
On 2022/05/08 6:45, Akira Ajisaka wrote:
Hi Chao,
How about using
https://repository.apache.org/content/repositories/orgapachehadoop-1348/
instead of https://repository.apache.org/content/repositories/staging/ ?
Akira
On Sat, May 7, 2022 at 10:52 AM Ayush Saxena <ayush...@gmail.com> wrote:
Hmm, I see the artifacts ideally should have got overwritten by the new
RC, but they didn’t. The reason seems like the staging path shared doesn’t
have any jars…
That is why it was picking the old jars. I think Steve needs to run mvn
deploy again…
Sent from my iPhone
On 07-May-2022, at 7:12 AM, Chao Sun <sunc...@apache.org> wrote:
Chao can you use the one that Steve mentioned in the mail?
Hmm how do I do that? Typically after closing the RC in nexus the
release bits will show up in
https://repository.apache.org/content/repositories/staging/org/apache/hadoop
and Spark build will be able to pick them up for testing. However in
this case I don't see any 3.3.3 jars in the URL.
On Fri, May 6, 2022 at 6:24 PM Ayush Saxena <ayush...@gmail.com> wrote:
There were two 3.3.3 staged. The earlier one was with skipShade, the
date was also april 22, I archived that. Chao can you use the one that
Steve mentioned in the mail?
On Sat, 7 May 2022 at 06:18, Chao Sun <sunc...@apache.org> wrote:
Seems there are some issues with the shaded client as I was not able
to compile Apache Spark with the RC
(https://github.com/apache/spark/pull/36474). Looks like it's compiled
with the `-DskipShade` option and the hadoop-client-api JAR doesn't
contain any class:
➜ hadoop-client-api jar tf 3.3.3/hadoop-client-api-3.3.3.jar
META-INF/
META-INF/MANIFEST.MF
META-INF/NOTICE.txt
META-INF/LICENSE.txt
META-INF/maven/
META-INF/maven/org.apache.hadoop/
META-INF/maven/org.apache.hadoop/hadoop-client-api/
META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.xml
META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.properties
On Fri, May 6, 2022 at 4:24 PM Stack <st...@duboce.net> wrote:
+1 (binding)
* Signature: ok
* Checksum : passed
* Rat check (1.8.0_191): passed
- mvn clean apache-rat:check
* Built from source (1.8.0_191): failed
- mvn clean install -DskipTests
- mvn -fae --no-transfer-progress -DskipTests
-Dmaven.javadoc.skip=true
-Pnative -Drequire.openssl -Drequire.snappy -Drequire.valgrind
-Drequire.zstd -Drequire.test.libhadoop clean install
* Unit tests pass (1.8.0_191):
- HDFS Tests passed (Didn't run more than this).
Deployed a ten node ha hdfs cluster with three namenodes and five
journalnodes. Ran a ten node hbase (older version of 2.5 branch built
against 3.3.2) against it. Tried a small verification job. Good. Ran a
bigger job with mild chaos. All seems to be working properly
(recoveries,
logs look fine). Killed a namenode. Failover worked promptly. UIs look
good. Poked at the hdfs cli. Seems good.
S
On Tue, May 3, 2022 at 4:24 AM Steve Loughran
<ste...@cloudera.com.invalid>
wrote:
I have put together a release candidate (rc0) for Hadoop 3.3.3
The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/
The git tag is release-3.3.3-RC0, commit d37586cbda3
The maven artifacts are staged at
https://repository.apache.org/content/repositories/orgapachehadoop-1348/
You can find my public key at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
Change log
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md
Release notes
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md
There's a very small number of changes, primarily critical
code/packaging
issues and security fixes.
- The critical fixes which shipped in the 3.2.3 release.
- CVEs in our code and dependencies
- Shaded client packaging issues.
- A switch from log4j to reload4j
reload4j is an active fork of the log4j 1.17 library with the
classes which
contain CVEs removed. Even though hadoop never used those classes,
they
regularly raised alerts on security scans and concen from users.
Switching
to the forked project allows us to ship a secure logging framework.
It will
complicate the builds of downstream maven/ivy/gradle projects which
exclude
our log4j artifacts, as they need to cut the new dependency
instead/as
well.
See the release notes for details.
This is my first release through the new docker build process, do
please
validate artifact signing &c to make sure it is good. I'll be trying
builds
of downstream projects.
We know there are some outstanding issues with at least one library
we are
shipping (okhttp), but I don't want to hold this release up for it.
If the
docker based release process works smoothly enough we can do a
followup
security release in a few weeks.
Please try the release and vote. The vote will run for 5 days.
-Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
