Hi @Ahmar Suhail <[email protected]> what about trying to upload the full tarbar again? Infra team has changed the quota now. Thus we could close this 3.4.2 release. Thanks.
Best Regards, - He Xiaoqiao On Fri, Sep 19, 2025 at 9:11 PM Steve Loughran <[email protected]> wrote: > > I have a WiP PR to pull all the object store artifacts into > share/common/lib; once we have the basic build working I"m going to extend > the dist profile with an aws-sdk profile that includes the AWS SDK. > > > 1. our public builds will skip the bundle > 2. s3a, abfs gs and other connectors will be on the classpath! > 3. for s3a, you'll need to add any version of the bundle.jar to > common/lib > > we wouldn't have a full/lean tar up online, just a single distro without > the bundle.jar file and a doc saying "download from X and drop into Y. We > tested using > version Z, later versions may nor may not work. " > > please review > https://github.com/apache/hadoop/pull/7980 > > > > On Fri, 19 Sept 2025 at 13:10, Suhail, Ahmar <[email protected]> wrote: > >> Thanks Shilun, >> >> >> >> I agree, For 3.4.2, we can move these projects to use the lean-tar. >> >> >> >> Separately, working with Steve to update the packaging so the full tar >> excludes the AWS bundle by default, so from the next release onwards, we >> will just have the full tars, and will no longer need the lean-tar. >> >> >> >> So two options for now: >> >> >> >> - We add a new hadoop-3.4.2.tar.gz, which is just a copy of >> hadoop-3.4.2-lean.tar.gz. >> (I would prefer not to do this as this requires some manual changes to the >> release directory) >> - We update Hive and Ozone to use the hadoop-3.4.2-lean.tar.gz for >> now. And from the next release, can go back to using hadoop-3.4.2.tar.gz. >> >> >> >> What do others think? >> >> >> >> *From: *slfan1989 <[email protected]> >> *Date: *Friday, 19 September 2025 at 04:20 >> *To: *"Suhail, Ahmar" <[email protected]> >> *Cc: *"Suhail, Ahmar" <[email protected]>, "[email protected]" >> <[email protected]>, Ayush Saxena <[email protected]>, Xiaoqiao He < >> [email protected]>, Steve Loughran <[email protected]>, Chris >> Nauroth <[email protected]>, Hadoop Common < >> [email protected]>, yarn-dev <[email protected]>, >> mapreduce-dev <[email protected]>, Hdfs-dev < >> [email protected]>, "<[email protected]>" < >> [email protected]> >> *Subject: *RE: [EXTERNAL] Missing full tarball for Hadoop 3.4.2 >> >> >> >> *CAUTION*: This email originated from outside of the organization. Do >> not click links or open attachments unless you can confirm the sender and >> know the content is safe. >> >> >> >> Thank you for your feedback. I think we do need to make a collective >> decision regarding the uploaded packages. At present, the absence of the >> full tar package prevents Ozone and Hive from using the latest Hadoop 3.4.2 >> release. From my perspective, using the minimal package as a substitute >> for the full package could be a reasonable option. >> >> >> >> Best regards, >> Shilun Fan. >> >> >> >> On Wed, Sep 10, 2025 at 5:56 PM Suhail, Ahmar <[email protected]> >> wrote: >> >> Hey all, >> >> Heard back from Infra: >> >> "The 1GB increase in INFRA-25423 was temporary. Your project needs to >> reduce the size of these binaries. In the previous temporary space >> increase, your project stated they would be removing extraneous >> dependencies/SDKs. We cannot keep increasing this allocation forever. What >> is your plan for reducing the object size? Packages of this size are >> difficult for users to download, and increase our bandwidth costs." >> >> So they're not too keen on increasing our limit. What do we want to do? >> We could rename the lean tars to hadoop-3.4.2.tar.gz, the consequence is >> that anyone downloading and trying to use S3A will hit errors, which might >> be ok as our release notes say the release no longer includes the SDK. >> >> >> On 01/09/2025, 09:40, "Suhail, Ahmar" <[email protected] <mailto: >> [email protected]>LID> wrote: >> >> >> CAUTION: This email originated from outside of the organization. Do not >> click links or open attachments unless you can confirm the sender and know >> the content is safe. >> >> >> >> >> >> >> Thanks Shilun, >> >> >> I’ve cut: https://issues.apache.org/jira/browse/INFRA-27182 < >> https://issues.apache.org/jira/browse/INFRA-27182>. >> >> >> Will upload the full tarball once this is resolved. >> >> >> From: slfan1989 <[email protected] <mailto:[email protected]>> >> Date: Saturday, 30 August 2025 at 09:29 >> To: "[email protected] <mailto:[email protected]>" <[email protected] >> <mailto:[email protected]>> >> Cc: Ayush Saxena <[email protected] <mailto:[email protected]>>, >> Xiaoqiao He <[email protected] <mailto:[email protected]>>, >> Steve Loughran <[email protected] <mailto:[email protected]>>, Chris >> Nauroth <[email protected] <mailto:[email protected]>>, Hadoop >> Common <[email protected] <mailto:[email protected]>>, >> yarn-dev <[email protected] <mailto:[email protected]>>, >> mapreduce-dev <[email protected] <mailto: >> [email protected]>>, Hdfs-dev <[email protected] >> <mailto:[email protected]>>, "<[email protected] >> <mailto:[email protected]>>" <[email protected] <mailto: >> [email protected]>> >> Subject: [EXTERNAL] Missing full tarball for Hadoop 3.4.2 >> >> >> >> >> CAUTION: This email originated from outside of the organization. Do not >> click links or open attachments unless you can confirm the sender and know >> the content is safe. >> >> >> Hi all, >> >> >> First of all, thanks to everyone for the efforts that led to the >> successful release of Hadoop 3.4.2. >> >> >> While trying to use the new dependency in the Hive project, Ayush noticed >> that the full Hadoop 3.4.2 tarball was not uploaded. >> >> >> We can find the reference discussion here: >> https://github.com/apache/hive/pull/6049 < >> https://github.com/apache/hive/pull/6049> >> >> >> After discussing with Ahmar, we plan to report this issue to INFRA, since >> they previously helped us by temporarily extending the storage quota for >> the release directory. >> >> >> Here is the JIRA I have submitted: >> https://issues.apache.org/jira/projects/INFRA/issues/INFRA-25423 < >> https://issues.apache.org/jira/projects/INFRA/issues/INFRA-25423> >> >> >> Best regards, >> Shilun Fan. >> >> >>
