Re: [DISCUSS] Enabling all platform builds in CI for all Hadoop PRs

Hi all, Although validating cross platform builds at pre-commit would be the most effective approach, I understand the huge disadvantage caused by the slowdown. The best way to tackle this would be to enable parallel builds for the different platforms. I had given it a shot about a year ago[1], it

[jira] [Reopened] (HDFS-16570) RBF: The router using MultipleDestinationMountTableResolver remove Multiple subcluster data under the mount point failed

[ https://issues.apache.org/jira/browse/HDFS-16570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiping Zhang reopened HDFS-16570: - > RBF: The router using MultipleDestinationMountTableResolver remove Multiple > subcluster data unde

Re: [VOTE] Release Apache Hadoop 3.3.3

Hmm, I see the artifacts ideally should have got overwritten by the new RC, but they didn’t. The reason seems like the staging path shared doesn’t have any jars… That is why it was picking the old jars. I think Steve needs to run mvn deploy again… Sent from my iPhone > On 07-May-2022, at 7:12

Re: [VOTE] Release Apache Hadoop 3.3.3

> Chao can you use the one that Steve mentioned in the mail? Hmm how do I do that? Typically after closing the RC in nexus the release bits will show up in https://repository.apache.org/content/repositories/staging/org/apache/hadoop and Spark build will be able to pick them up for testing. However

Re: [VOTE] Release Apache Hadoop 3.3.3

There were two 3.3.3 staged. The earlier one was with skipShade, the date was also april 22, I archived that. Chao can you use the one that Steve mentioned in the mail? On Sat, 7 May 2022 at 06:18, Chao Sun wrote: > Seems there are some issues with the shaded client as I was not able > to compil

Re: [VOTE] Release Apache Hadoop 3.3.3

Seems there are some issues with the shaded client as I was not able to compile Apache Spark with the RC (https://github.com/apache/spark/pull/36474). Looks like it's compiled with the `-DskipShade` option and the hadoop-client-api JAR doesn't contain any class: ➜ hadoop-client-api jar tf 3.3.3/h

[jira] [Resolved] (HDFS-16570) RBF: The router using MultipleDestinationMountTableResolver remove Multiple subcluster data under the mount point failed

[ https://issues.apache.org/jira/browse/HDFS-16570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiping Zhang resolved HDFS-16570. - Resolution: Implemented > RBF: The router using MultipleDestinationMountTableResolver remove Mult

Re: [VOTE] Release Apache Hadoop 3.3.3

+1 (binding) * Signature: ok * Checksum : passed * Rat check (1.8.0_191): passed - mvn clean apache-rat:check * Built from source (1.8.0_191): failed - mvn clean install -DskipTests - mvn -fae --no-transfer-progress -DskipTests -Dmaven.javadoc.skip=true -Pnative -Drequire.openssl

Re: [VOTE] Release Apache Hadoop 3.3.3

+1, * Built from source * Successful native build on Ubuntu 18.04 * Verified Checksums (CHANGELOG.md,RELEASENOTES.md,hadoop-3.3.3-rat.txt,hadoop-3.3.3-site.tar.gz,hadoop-3.3.3-src.tar.gz,hadoop-3.3.3.tar.gz) * Successful RAT check * Ran some basic HDFS shell commands * Ran some basic YARN shell com

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64

For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/861/ No changes -1 overall The following subsystems voted -1: blanks pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javado

Re: [VOTE] Release Apache Hadoop 3.3.3

+1 (non-binding), With a minor change in hadoop-vote, * Signature: ok * Checksum : ok * Rat check (1.8.0_301): ok - mvn clean apache-rat:check * Built from source (1.8.0_301): ok - mvn clean install -DskipTests * Built tar from source (1.8.0_301): o

[jira] [Resolved] (HDFS-16520) Improve EC pread: avoid potential reading whole block

[ https://issues.apache.org/jira/browse/HDFS-16520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-16520. Resolution: Fixed Merged the PR and cherrypicked into branch-3.3. Thanks! > Improve EC pr

[jira] [Created] (HDFS-16571) ABFS: Two BlobCreated get triggered for writing one ABFS file

Bryan Chen created HDFS-16571: - Summary: ABFS: Two BlobCreated get triggered for writing one ABFS file Key: HDFS-16571 URL: https://issues.apache.org/jira/browse/HDFS-16571 Project: Hadoop HDFS

Re: [DISCUSS] Enabling all platform builds in CI for all Hadoop PRs

>From functional point of view it does makes sense to validate all the platforms as part of the builds, but the Pre commits builds taking time is now no longer a small things, In past one year or may be two, we have already increased it more than twice as compared to what it was before, If someone

Re: [DISCUSS] Enabling all platform builds in CI for all Hadoop PRs

Running builds for all platforms for each and every PR seems too excessive. How about doing all platform builds in the nightly jobs? On Fri, May 6, 2022 at 8:02 AM Steve Loughran wrote: > I'm not enthusiastic here as it not only makes the builds slower, it > reduces the #of builds we can throug

Re: [DISCUSS] Enabling all platform builds in CI for all Hadoop PRs

I'm not enthusiastic here as it not only makes the builds slower, it reduces the #of builds we can through a day one thing I am wondering is could we remove java8 support on some branches? make branch 3.3.2.x (i.e the 3.3.3 release) the last java 8 build, and this summers branch-3.3 release (whic

Re: [VOTE] Release Apache Hadoop 3.3.3

+1 (binding) * verified signature and checksum of the source tarball. * built the source code on Rocky Linux 8 (x86_64) and OpenJDK 8 by `mvn install -DskipTests -Pnative -Pdist`. * launched pseudo distributed cluster with Kerberos security enabled and ran sample MR jobs. * launched HA enabled

[jira] [Created] (HDFS-16570) RBF: The router using MultipleDestinationMountTableResolver remove Multiple subcluster data under the mount point failed

Xiping Zhang created HDFS-16570: --- Summary: RBF: The router using MultipleDestinationMountTableResolver remove Multiple subcluster data under the mount point failed Key: HDFS-16570 URL: https://issues.apache.org/jir

Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64

For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/653/ No changes -1 overall The following subsystems voted -1: asflicense hadolint mvnsite pathlen unit The following subsystems voted -1 but were configured to be filtered/ignored: cc