Heads-up that I brought "Magic Committer optional clean up" into branch-3.4.2 for the next RC, as there was a desire to get it into a 3.4 release.
https://issues.apache.org/jira/browse/HADOOP-18568 https://github.com/apache/hadoop/pull/7693 Chris Nauroth On Wed, Jun 11, 2025 at 9:48 AM Steve Loughran <ste...@cloudera.com.invalid> wrote: > this is my PR for parquet buffers > https://github.com/apache/hadoop/pull/7732 > > doesn't do any complex work, just > * adds a capability to declare buffers may be sliced > * adds a switch to turn verification off for file:// > * adds tests to show that slicing may happen > * docs > > For Parquet I'm fixing the tests just by deleting the .crc files; will > propose rolling back enabling vector IO PR and document it. > > Vector reading is essential for s3a perf unless you use the analytics > stream, so people should turn it on there. for local FS without checksums, > it is really good too as it uses async IO for parallel reads. No difference > for other stores. I'm not going to turn it off. > > > > On Mon, 9 Jun 2025 at 16:13, Cheng Pan <pan3...@gmail.com> wrote: > > > Thanks for kicking off this release. > > > > I integrated it with Spark master branch[1], no surprise so far. Looking > > forward to the next RC. > > > > [1] https://github.com/apache/spark/pull/51127 > > > > Thanks, > > Cheng Pan > > > > On 2025/05/28 12:24:52 Ahmar Suhail wrote: > > > Hey all, > > > > > > The first release candidate for Hadoop 3.4.2 is now available for > voting. > > > > > > There are a couple of things to note: > > > > > > 1/ No Arm64 artifacts. This is due to previously reported issues: > > > https://issues.apache.org/jira/projects/YARN/issues/YARN-11712 and > > > ttps://issues.apache.org/jira/projects/YARN/issues/YARN-11713 > > > <https://issues.apache.org/jira/projects/YARN/issues/YARN-11713>, > which > > > mean that the build fails on arm64. > > > > > > 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle > > > from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB, > > > which makes the size of tar > 1GB, and it can no longer be uploaded to > > SVN. > > > For S3A, download SDK bundle v2.29.52 from: > > > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 > > , > > > and drop it into /share/hadoop/common/lib. Release notes will be > updated > > > with these instructions. > > > > > > > > > The RC is available at: > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.4.2-RC1/ > > > > > > The git tag is release-3.4.2-RC1, commit > > > 09870840ec35b48cd107972eb24d25e8aece04c9 > > > > > > The maven artifacts are staged at: > > > > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1437 > > > > > > > > > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) > at: > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > > > > > > This release has been created off of branch-3.4. Key changes include: > > > > > > * S3A: Integration with S3 Analytics Accelerator input stream > > > * S3A: Support for S3 conditional writes > > > * ABFS: Deprecation of WASB driver > > > * ABFS: Support for Non-Heirarchical Namespace Accounts on ABFS Driver > > > > > > > > > This is my first attempt at managing a release, please do test the > > release > > > and let me know in case of any issues. > > > > > > Thanks, > > > Ahmar > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > > > >
