Re: [DISCUSS] Enabling all platform builds in CI for all Hadoop PRs

Hi all, Although validating cross platform builds at pre-commit would be the most effective approach, I understand the huge disadvantage caused by the slowdown. The best way to tackle this would be to enable parallel builds for the different platforms. I had given it a shot about a year ago[1], it

Re: [VOTE] Release Apache Hadoop 3.3.3

Hmm, I see the artifacts ideally should have got overwritten by the new RC, but they didn’t. The reason seems like the staging path shared doesn’t have any jars… That is why it was picking the old jars. I think Steve needs to run mvn deploy again… Sent from my iPhone > On 07-May-2022, at 7:12

Re: [VOTE] Release Apache Hadoop 3.3.3

> Chao can you use the one that Steve mentioned in the mail? Hmm how do I do that? Typically after closing the RC in nexus the release bits will show up in https://repository.apache.org/content/repositories/staging/org/apache/hadoop and Spark build will be able to pick them up for testing. However

Re: [VOTE] Release Apache Hadoop 3.3.3

There were two 3.3.3 staged. The earlier one was with skipShade, the date was also april 22, I archived that. Chao can you use the one that Steve mentioned in the mail? On Sat, 7 May 2022 at 06:18, Chao Sun wrote: > Seems there are some issues with the shaded client as I was not able > to compil

Re: [VOTE] Release Apache Hadoop 3.3.3

Seems there are some issues with the shaded client as I was not able to compile Apache Spark with the RC (https://github.com/apache/spark/pull/36474). Looks like it's compiled with the `-DskipShade` option and the hadoop-client-api JAR doesn't contain any class: ➜ hadoop-client-api jar tf 3.3.3/h

Re: [VOTE] Release Apache Hadoop 3.3.3

+1 (binding) * Signature: ok * Checksum : passed * Rat check (1.8.0_191): passed - mvn clean apache-rat:check * Built from source (1.8.0_191): failed - mvn clean install -DskipTests - mvn -fae --no-transfer-progress -DskipTests -Dmaven.javadoc.skip=true -Pnative -Drequire.openssl

Re: [VOTE] Release Apache Hadoop 3.3.3

+1, * Built from source * Successful native build on Ubuntu 18.04 * Verified Checksums (CHANGELOG.md,RELEASENOTES.md,hadoop-3.3.3-rat.txt,hadoop-3.3.3-site.tar.gz,hadoop-3.3.3-src.tar.gz,hadoop-3.3.3.tar.gz) * Successful RAT check * Ran some basic HDFS shell commands * Ran some basic YARN shell com

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64

For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/861/ No changes -1 overall The following subsystems voted -1: blanks pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javado

Re: [VOTE] Release Apache Hadoop 3.3.3

+1 (non-binding), With a minor change in hadoop-vote, * Signature: ok * Checksum : ok * Rat check (1.8.0_301): ok - mvn clean apache-rat:check * Built from source (1.8.0_301): ok - mvn clean install -DskipTests * Built tar from source (1.8.0_301): o

[jira] [Created] (HADOOP-18228) Update hadoop-vote to use HADOOP_RC_VERSION dir

Viraj Jasani created HADOOP-18228: - Summary: Update hadoop-vote to use HADOOP_RC_VERSION dir Key: HADOOP-18228 URL: https://issues.apache.org/jira/browse/HADOOP-18228 Project: Hadoop Common I

[jira] [Created] (HADOOP-18227) Add IOstats for vectored IO api.

Mukund Thakur created HADOOP-18227: -- Summary: Add IOstats for vectored IO api. Key: HADOOP-18227 URL: https://issues.apache.org/jira/browse/HADOOP-18227 Project: Hadoop Common Issue Type: Su

Re: [VOTE] Release Apache Hadoop 3.3.3

+1 (advisory) * Verified sha512 checksum was correct for source tarball * Verified signature was correct for source tarball (not verified trust) * Built source code from tarball on Ubuntu 20.04 (x86) and OpenJDK 8 in Amazon EC2 * Verified source tarball matches content of RC0 tag on GitHub * Veri

Re: Publishing hadoop-lzo maven package in com.hadoop namespace

oh certainly, that's always an option. And sounds like the one we'll pursue. On Fri, May 6, 2022 at 7:59 AM Steve Loughran wrote: > it;s com.hadoop, but still tainted. > > couldn't we have com.twitter.hadoop? > > On Fri, 6 May 2022 at 15:23, Sean Busbey > wrote: > > > As an ASF project we can

Re: [DISCUSS] Enabling all platform builds in CI for all Hadoop PRs

>From functional point of view it does makes sense to validate all the platforms as part of the builds, but the Pre commits builds taking time is now no longer a small things, In past one year or may be two, we have already increased it more than twice as compared to what it was before, If someone

Re: [DISCUSS] Enabling all platform builds in CI for all Hadoop PRs

Running builds for all platforms for each and every PR seems too excessive. How about doing all platform builds in the nightly jobs? On Fri, May 6, 2022 at 8:02 AM Steve Loughran wrote: > I'm not enthusiastic here as it not only makes the builds slower, it > reduces the #of builds we can throug

Re: [DISCUSS] Enabling all platform builds in CI for all Hadoop PRs

I'm not enthusiastic here as it not only makes the builds slower, it reduces the #of builds we can through a day one thing I am wondering is could we remove java8 support on some branches? make branch 3.3.2.x (i.e the 3.3.3 release) the last java 8 build, and this summers branch-3.3 release (whic

Re: Publishing hadoop-lzo maven package in com.hadoop namespace

it;s com.hadoop, but still tainted. couldn't we have com.twitter.hadoop? On Fri, 6 May 2022 at 15:23, Sean Busbey wrote: > As an ASF project we can not distribute GPL code. As an ASF project it’s > also our job to defend our trademarks. > > We should actively prevent this GPL code from going to

Re: Publishing hadoop-lzo maven package in com.hadoop namespace

As an ASF project we can not distribute GPL code. As an ASF project it’s also our job to defend our trademarks. We should actively prevent this GPL code from going to maven central with our name attached to it. > On May 5, 2022, at 1:49 PM, Wei-Chiu Chuang wrote: > > Please be careful. > So

Re: [VOTE] Release Apache Hadoop 3.3.3

+1 (binding) * verified signature and checksum of the source tarball. * built the source code on Rocky Linux 8 (x86_64) and OpenJDK 8 by `mvn install -DskipTests -Pnative -Pdist`. * launched pseudo distributed cluster with Kerberos security enabled and ran sample MR jobs. * launched HA enabled

Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64

For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/653/ No changes -1 overall The following subsystems voted -1: asflicense hadolint mvnsite pathlen unit The following subsystems voted -1 but were configured to be filtered/ignored: cc