Agreed, that step #10 is out-dated and should be removed (I skipped that when releasing Hadoop 3.3.2 but didn't update it, sorry).
> How about using > https://repository.apache.org/content/repositories/orgapachehadoop-1348/ Akira, I tried this too but it didn't work. I think we'd need the artifacts to be properly pushed to the staging repository. > Could you please let me know how I can consume the Hadoop 3 jars in maven? Gautham (if you are following this thread), you'll need to add the following: <repository> <id>staged</id> <name>staged-releases</name> <url>https://repository.apache.org/content/repositories/staging/</url> <releases> <enabled>true</enabled> </releases> <snapshots> <enabled>true</enabled> </snapshots> </repository> to the `<repositories>` section in your Maven pom.xml file. On Mon, May 9, 2022 at 8:52 AM Steve Loughran <ste...@cloudera.com.invalid> wrote: > > I didn't do that as the docker image was doing it itself...I discussed this > with Akira and Ayush & they agreed. so whatever went wrong. it was > something else. > > I have been building a list of things I'd like to change there; cutting > that line was one of them. but I need to work out the correct workflow. > > trying again, and creating a stub module to verify the client is in staging > > On Mon, 9 May 2022 at 15:19, Masatake Iwasaki <iwasak...@oss.nttdata.co.jp> > wrote: > > > It seems to be caused by obsolete instruction in HowToRelease Wiki? > > > > After HADOOP-15058, `mvn deploy` is kicked by > > `dev-support/bin/create-release --asfrelease`. > > https://issues.apache.org/jira/browse/HADOOP-15058 > > > > Step #10 in "Creating the release candidate (X.Y.Z-RC<N>)" section > > of the Wiki still instructs to run `mvn deploy` with `-DskipShade`. > > > > 2 sets of artifact are deployed after creating RC based on the instruction. > > The latest one contains empty shaded jars. > > > > hadoop-client-api and hadoop-client-runtime of already released 3.2.3 > > looks having same issue... > > > > Masatake Iwasaki > > > > On 2022/05/08 6:45, Akira Ajisaka wrote: > > > Hi Chao, > > > > > > How about using > > > https://repository.apache.org/content/repositories/orgapachehadoop-1348/ > > > instead of https://repository.apache.org/content/repositories/staging/ ? > > > > > > Akira > > > > > > On Sat, May 7, 2022 at 10:52 AM Ayush Saxena <ayush...@gmail.com> wrote: > > > > > >> Hmm, I see the artifacts ideally should have got overwritten by the new > > >> RC, but they didn’t. The reason seems like the staging path shared > > doesn’t > > >> have any jars… > > >> That is why it was picking the old jars. I think Steve needs to run mvn > > >> deploy again… > > >> > > >> Sent from my iPhone > > >> > > >>> On 07-May-2022, at 7:12 AM, Chao Sun <sunc...@apache.org> wrote: > > >>> > > >>> > > >>>> > > >>>> Chao can you use the one that Steve mentioned in the mail? > > >>> > > >>> Hmm how do I do that? Typically after closing the RC in nexus the > > >>> release bits will show up in > > >>> > > >> > > https://repository.apache.org/content/repositories/staging/org/apache/hadoop > > >>> and Spark build will be able to pick them up for testing. However in > > >>> this case I don't see any 3.3.3 jars in the URL. > > >>> > > >>>> On Fri, May 6, 2022 at 6:24 PM Ayush Saxena <ayush...@gmail.com> > > wrote: > > >>>> > > >>>> There were two 3.3.3 staged. The earlier one was with skipShade, the > > >> date was also april 22, I archived that. Chao can you use the one that > > >> Steve mentioned in the mail? > > >>>> > > >>>>> On Sat, 7 May 2022 at 06:18, Chao Sun <sunc...@apache.org> wrote: > > >>>>> > > >>>>> Seems there are some issues with the shaded client as I was not able > > >>>>> to compile Apache Spark with the RC > > >>>>> (https://github.com/apache/spark/pull/36474). Looks like it's > > compiled > > >>>>> with the `-DskipShade` option and the hadoop-client-api JAR doesn't > > >>>>> contain any class: > > >>>>> > > >>>>> ➜ hadoop-client-api jar tf 3.3.3/hadoop-client-api-3.3.3.jar > > >>>>> META-INF/ > > >>>>> META-INF/MANIFEST.MF > > >>>>> META-INF/NOTICE.txt > > >>>>> META-INF/LICENSE.txt > > >>>>> META-INF/maven/ > > >>>>> META-INF/maven/org.apache.hadoop/ > > >>>>> META-INF/maven/org.apache.hadoop/hadoop-client-api/ > > >>>>> META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.xml > > >>>>> META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.properties > > >>>>> > > >>>>> On Fri, May 6, 2022 at 4:24 PM Stack <st...@duboce.net> wrote: > > >>>>>> > > >>>>>> +1 (binding) > > >>>>>> > > >>>>>> * Signature: ok > > >>>>>> * Checksum : passed > > >>>>>> * Rat check (1.8.0_191): passed > > >>>>>> - mvn clean apache-rat:check > > >>>>>> * Built from source (1.8.0_191): failed > > >>>>>> - mvn clean install -DskipTests > > >>>>>> - mvn -fae --no-transfer-progress -DskipTests > > >> -Dmaven.javadoc.skip=true > > >>>>>> -Pnative -Drequire.openssl -Drequire.snappy -Drequire.valgrind > > >>>>>> -Drequire.zstd -Drequire.test.libhadoop clean install > > >>>>>> * Unit tests pass (1.8.0_191): > > >>>>>> - HDFS Tests passed (Didn't run more than this). > > >>>>>> > > >>>>>> Deployed a ten node ha hdfs cluster with three namenodes and five > > >>>>>> journalnodes. Ran a ten node hbase (older version of 2.5 branch > > built > > >>>>>> against 3.3.2) against it. Tried a small verification job. Good. > > Ran a > > >>>>>> bigger job with mild chaos. All seems to be working properly > > >> (recoveries, > > >>>>>> logs look fine). Killed a namenode. Failover worked promptly. UIs > > look > > >>>>>> good. Poked at the hdfs cli. Seems good. > > >>>>>> > > >>>>>> S > > >>>>>> > > >>>>>> On Tue, May 3, 2022 at 4:24 AM Steve Loughran > > >> <ste...@cloudera.com.invalid> > > >>>>>> wrote: > > >>>>>> > > >>>>>>> I have put together a release candidate (rc0) for Hadoop 3.3.3 > > >>>>>>> > > >>>>>>> The RC is available at: > > >>>>>>> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/ > > >>>>>>> > > >>>>>>> The git tag is release-3.3.3-RC0, commit d37586cbda3 > > >>>>>>> > > >>>>>>> The maven artifacts are staged at > > >>>>>>> > > >> > > https://repository.apache.org/content/repositories/orgapachehadoop-1348/ > > >>>>>>> > > >>>>>>> You can find my public key at: > > >>>>>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > >>>>>>> > > >>>>>>> Change log > > >>>>>>> > > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md > > >>>>>>> > > >>>>>>> Release notes > > >>>>>>> > > >> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md > > >>>>>>> > > >>>>>>> There's a very small number of changes, primarily critical > > >> code/packaging > > >>>>>>> issues and security fixes. > > >>>>>>> > > >>>>>>> > > >>>>>>> - The critical fixes which shipped in the 3.2.3 release. > > >>>>>>> - CVEs in our code and dependencies > > >>>>>>> - Shaded client packaging issues. > > >>>>>>> - A switch from log4j to reload4j > > >>>>>>> > > >>>>>>> > > >>>>>>> reload4j is an active fork of the log4j 1.17 library with the > > >> classes which > > >>>>>>> contain CVEs removed. Even though hadoop never used those classes, > > >> they > > >>>>>>> regularly raised alerts on security scans and concen from users. > > >> Switching > > >>>>>>> to the forked project allows us to ship a secure logging framework. > > >> It will > > >>>>>>> complicate the builds of downstream maven/ivy/gradle projects which > > >> exclude > > >>>>>>> our log4j artifacts, as they need to cut the new dependency > > >> instead/as > > >>>>>>> well. > > >>>>>>> > > >>>>>>> See the release notes for details. > > >>>>>>> > > >>>>>>> This is my first release through the new docker build process, do > > >> please > > >>>>>>> validate artifact signing &c to make sure it is good. I'll be > > trying > > >> builds > > >>>>>>> of downstream projects. > > >>>>>>> > > >>>>>>> We know there are some outstanding issues with at least one library > > >> we are > > >>>>>>> shipping (okhttp), but I don't want to hold this release up for it. > > >> If the > > >>>>>>> docker based release process works smoothly enough we can do a > > >> followup > > >>>>>>> security release in a few weeks. > > >>>>>>> > > >>>>>>> Please try the release and vote. The vote will run for 5 days. > > >>>>>>> > > >>>>>>> -Steve > > >>>>>>> > > >>>>> > > >>>>> --------------------------------------------------------------------- > > >>>>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > >>>>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > >>>>> > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > > >> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > >> > > >> > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
