I've done another docker build and the client jars appear to be there. I'll test tomorrow before putting up another vote. it will be exactly the same commit as before, just recompiled/republished
On Mon, 9 May 2022 at 17:45, Chao Sun <sunc...@apache.org> wrote: > Agreed, that step #10 is out-dated and should be removed (I skipped > that when releasing Hadoop 3.3.2 but didn't update it, sorry). > > > How about using > https://repository.apache.org/content/repositories/orgapachehadoop-1348/ > > Akira, I tried this too but it didn't work. I think we'd need the > artifacts to be properly pushed to the staging repository. > > > Could you please let me know how I can consume the Hadoop 3 jars in > maven? > > Gautham (if you are following this thread), you'll need to add the > following: > > <repository> > <id>staged</id> > <name>staged-releases</name> > <url>https://repository.apache.org/content/repositories/staging/ > </url> > <releases> > <enabled>true</enabled> > </releases> > <snapshots> > <enabled>true</enabled> > </snapshots> > </repository> > > to the `<repositories>` section in your Maven pom.xml file. > > On Mon, May 9, 2022 at 8:52 AM Steve Loughran > <ste...@cloudera.com.invalid> wrote: > > > > I didn't do that as the docker image was doing it itself...I discussed > this > > with Akira and Ayush & they agreed. so whatever went wrong. it was > > something else. > > > > I have been building a list of things I'd like to change there; cutting > > that line was one of them. but I need to work out the correct workflow. > > > > trying again, and creating a stub module to verify the client is in > staging > > > > On Mon, 9 May 2022 at 15:19, Masatake Iwasaki < > iwasak...@oss.nttdata.co.jp> > > wrote: > > > > > It seems to be caused by obsolete instruction in HowToRelease Wiki? > > > > > > After HADOOP-15058, `mvn deploy` is kicked by > > > `dev-support/bin/create-release --asfrelease`. > > > https://issues.apache.org/jira/browse/HADOOP-15058 > > > > > > Step #10 in "Creating the release candidate (X.Y.Z-RC<N>)" section > > > of the Wiki still instructs to run `mvn deploy` with `-DskipShade`. > > > > > > 2 sets of artifact are deployed after creating RC based on the > instruction. > > > The latest one contains empty shaded jars. > > > > > > hadoop-client-api and hadoop-client-runtime of already released 3.2.3 > > > looks having same issue... > > > > > > Masatake Iwasaki > > > > > > On 2022/05/08 6:45, Akira Ajisaka wrote: > > > > Hi Chao, > > > > > > > > How about using > > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1348/ > > > > instead of > https://repository.apache.org/content/repositories/staging/ ? > > > > > > > > Akira > > > > > > > > On Sat, May 7, 2022 at 10:52 AM Ayush Saxena <ayush...@gmail.com> > wrote: > > > > > > > >> Hmm, I see the artifacts ideally should have got overwritten by the > new > > > >> RC, but they didn’t. The reason seems like the staging path shared > > > doesn’t > > > >> have any jars… > > > >> That is why it was picking the old jars. I think Steve needs to run > mvn > > > >> deploy again… > > > >> > > > >> Sent from my iPhone > > > >> > > > >>> On 07-May-2022, at 7:12 AM, Chao Sun <sunc...@apache.org> wrote: > > > >>> > > > >>> > > > >>>> > > > >>>> Chao can you use the one that Steve mentioned in the mail? > > > >>> > > > >>> Hmm how do I do that? Typically after closing the RC in nexus the > > > >>> release bits will show up in > > > >>> > > > >> > > > > https://repository.apache.org/content/repositories/staging/org/apache/hadoop > > > >>> and Spark build will be able to pick them up for testing. However > in > > > >>> this case I don't see any 3.3.3 jars in the URL. > > > >>> > > > >>>> On Fri, May 6, 2022 at 6:24 PM Ayush Saxena <ayush...@gmail.com> > > > wrote: > > > >>>> > > > >>>> There were two 3.3.3 staged. The earlier one was with skipShade, > the > > > >> date was also april 22, I archived that. Chao can you use the one > that > > > >> Steve mentioned in the mail? > > > >>>> > > > >>>>> On Sat, 7 May 2022 at 06:18, Chao Sun <sunc...@apache.org> > wrote: > > > >>>>> > > > >>>>> Seems there are some issues with the shaded client as I was not > able > > > >>>>> to compile Apache Spark with the RC > > > >>>>> (https://github.com/apache/spark/pull/36474). Looks like it's > > > compiled > > > >>>>> with the `-DskipShade` option and the hadoop-client-api JAR > doesn't > > > >>>>> contain any class: > > > >>>>> > > > >>>>> ➜ hadoop-client-api jar tf 3.3.3/hadoop-client-api-3.3.3.jar > > > >>>>> META-INF/ > > > >>>>> META-INF/MANIFEST.MF > > > >>>>> META-INF/NOTICE.txt > > > >>>>> META-INF/LICENSE.txt > > > >>>>> META-INF/maven/ > > > >>>>> META-INF/maven/org.apache.hadoop/ > > > >>>>> META-INF/maven/org.apache.hadoop/hadoop-client-api/ > > > >>>>> META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.xml > > > >>>>> META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.properties > > > >>>>> > > > >>>>> On Fri, May 6, 2022 at 4:24 PM Stack <st...@duboce.net> wrote: > > > >>>>>> > > > >>>>>> +1 (binding) > > > >>>>>> > > > >>>>>> * Signature: ok > > > >>>>>> * Checksum : passed > > > >>>>>> * Rat check (1.8.0_191): passed > > > >>>>>> - mvn clean apache-rat:check > > > >>>>>> * Built from source (1.8.0_191): failed > > > >>>>>> - mvn clean install -DskipTests > > > >>>>>> - mvn -fae --no-transfer-progress -DskipTests > > > >> -Dmaven.javadoc.skip=true > > > >>>>>> -Pnative -Drequire.openssl -Drequire.snappy -Drequire.valgrind > > > >>>>>> -Drequire.zstd -Drequire.test.libhadoop clean install > > > >>>>>> * Unit tests pass (1.8.0_191): > > > >>>>>> - HDFS Tests passed (Didn't run more than this). > > > >>>>>> > > > >>>>>> Deployed a ten node ha hdfs cluster with three namenodes and > five > > > >>>>>> journalnodes. Ran a ten node hbase (older version of 2.5 branch > > > built > > > >>>>>> against 3.3.2) against it. Tried a small verification job. Good. > > > Ran a > > > >>>>>> bigger job with mild chaos. All seems to be working properly > > > >> (recoveries, > > > >>>>>> logs look fine). Killed a namenode. Failover worked promptly. > UIs > > > look > > > >>>>>> good. Poked at the hdfs cli. Seems good. > > > >>>>>> > > > >>>>>> S > > > >>>>>> > > > >>>>>> On Tue, May 3, 2022 at 4:24 AM Steve Loughran > > > >> <ste...@cloudera.com.invalid> > > > >>>>>> wrote: > > > >>>>>> > > > >>>>>>> I have put together a release candidate (rc0) for Hadoop 3.3.3 > > > >>>>>>> > > > >>>>>>> The RC is available at: > > > >>>>>>> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/ > > > >>>>>>> > > > >>>>>>> The git tag is release-3.3.3-RC0, commit d37586cbda3 > > > >>>>>>> > > > >>>>>>> The maven artifacts are staged at > > > >>>>>>> > > > >> > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1348/ > > > >>>>>>> > > > >>>>>>> You can find my public key at: > > > >>>>>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > >>>>>>> > > > >>>>>>> Change log > > > >>>>>>> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md > > > >>>>>>> > > > >>>>>>> Release notes > > > >>>>>>> > > > >> > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md > > > >>>>>>> > > > >>>>>>> There's a very small number of changes, primarily critical > > > >> code/packaging > > > >>>>>>> issues and security fixes. > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> - The critical fixes which shipped in the 3.2.3 release. > > > >>>>>>> - CVEs in our code and dependencies > > > >>>>>>> - Shaded client packaging issues. > > > >>>>>>> - A switch from log4j to reload4j > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> reload4j is an active fork of the log4j 1.17 library with the > > > >> classes which > > > >>>>>>> contain CVEs removed. Even though hadoop never used those > classes, > > > >> they > > > >>>>>>> regularly raised alerts on security scans and concen from > users. > > > >> Switching > > > >>>>>>> to the forked project allows us to ship a secure logging > framework. > > > >> It will > > > >>>>>>> complicate the builds of downstream maven/ivy/gradle projects > which > > > >> exclude > > > >>>>>>> our log4j artifacts, as they need to cut the new dependency > > > >> instead/as > > > >>>>>>> well. > > > >>>>>>> > > > >>>>>>> See the release notes for details. > > > >>>>>>> > > > >>>>>>> This is my first release through the new docker build process, > do > > > >> please > > > >>>>>>> validate artifact signing &c to make sure it is good. I'll be > > > trying > > > >> builds > > > >>>>>>> of downstream projects. > > > >>>>>>> > > > >>>>>>> We know there are some outstanding issues with at least one > library > > > >> we are > > > >>>>>>> shipping (okhttp), but I don't want to hold this release up > for it. > > > >> If the > > > >>>>>>> docker based release process works smoothly enough we can do a > > > >> followup > > > >>>>>>> security release in a few weeks. > > > >>>>>>> > > > >>>>>>> Please try the release and vote. The vote will run for 5 days. > > > >>>>>>> > > > >>>>>>> -Steve > > > >>>>>>> > > > >>>>> > > > >>>>> > --------------------------------------------------------------------- > > > >>>>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > > >>>>> For additional commands, e-mail: > common-dev-h...@hadoop.apache.org > > > >>>>> > > > >> > > > >> > --------------------------------------------------------------------- > > > >> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > > > >> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > > >> > > > >> > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > > > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > > > > > > >
