[RESULT] [VOTE] Release Apache Hadoop Thirdparty 1.3.0

2024-08-27 Thread Steve Loughran
With 5 binding +1 votes and 0 negative votes, the hadoop-thirdparty release vote was successful I'm going to complete the release today, after which mukund can do another RC of the 3.4.1 release Thanks to everyone who tested it! -steve

Re: [VOTE] Release Apache Hadoop Thirdparty 1.3.0

2024-08-22 Thread Steve Loughran
utions, > please read the following articles: > [ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException > [ERROR] > [ERROR] After correcting the problems, you can resume the build with the > command > [ERROR] mvn -rf :hadoop-shaded-protobuf_3_25 &

[VOTE] Release Apache Hadoop Thirdparty 1.3.0

2024-08-20 Thread Steve Loughran
Mukund and I have built a release candidate (RC1) for Hadoop-Thirdparty 1.3.0. The RC is available at: https://dist.apache.org/repos/dist/dev/hadoop/hadoop-thirdparty-1.3.0-RC1/ The git tag is release-1.3.0-RC1, commit 0fd62903b071b5186f31b7030ce42e1c00f6bb6a The maven artifacts are staged at ht

[DISCUSS] what needs to be done to switch to java17

2024-07-29 Thread Steve Loughran
A lot of projects are moving off java8. making java17 the new baseline what do we need to there that is blocker rather than just "nice"'?

[DISCUSS] lots of yetus failures in hdfs

2024-06-06 Thread Steve Loughran
PR's which trigger hdfs builds seem to hit a lot of hdfs test failures https://github.com/apache/hadoop/pull/6675 Are these regressions or are the tests flaky? I don't want commit patches which break things, yet hdfs tests seem unreliable and so I'm dangerously tempted to +1 anyway...

Re: [DISCUSS] pruning surplus branches/tags from the main hadoop repo

2024-05-30 Thread Steve Loughran
ozone-0.3.0-alpha-RC1' (was cdad29240e52) >> Deleted tag 'ozone-0.4.0-alpha-RC0' (was 07fd26ef6d8c) >> Deleted tag 'ozone-0.4.0-alpha-RC1' (was c4f9a20bbe55) >> Deleted tag 'ozone-0.4.0-alpha-RC2' (was 6860c595ed19) >> Deleted tag 'ozone-0.4.

[DISCUSS] pruning surplus branches/tags from the main hadoop repo

2024-05-29 Thread Steve Loughran
I'm waiting for a git update of trunk to complete, not having done it since last week. The 1.8 GB download is taking a long time over a VPN. Updating files: 100% (8518/8518), done. Switched to branch 'trunk' Your branch is up to date with 'apache/trunk'. remote: Enumerating objects: 4142992, done.

Re: [VOTE] Release Apache Hadoop 3.4.0 (RC3)

2024-03-12 Thread Steve Loughran
d 5 affirmative votes, with 4 votes binding > and > > 1 vote non-binding. > > > > Thank you very much for voting and verifying! > > > > This voting will continue until March 15th, this Friday. > > > > Best Regards, > > Shilun Fan. > > > &

Re: [VOTE] Release Apache Hadoop 3.4.0 (RC3)

2024-03-11 Thread Steve Loughran
+1 binding (sorry, this had ended in the yarn-dev folder, otherwise I'd have seen it earlier. been testing it this afternoon: pulled the latest version of https://github.com/apache/hadoop-release-support (note, this module is commit-then-review; whoever is working on/validating a release can comm

Re: [DISCUSS] Support/Fate of HBase v1 in Hadoop

2024-03-11 Thread Steve Loughran
+1 for cutting hbase 1; it only reduces dependency pain (no more protobuf 2.5!) Created the JIRA on that a few days back https://issues.apache.org/jira/browse/YARN-11658 On Tue, 5 Mar 2024 at 12:08, Bryan Beaudreault wrote: > Hbase v1 is EOL for a while now, so option 2 probably makes sense. W

Re: [VOTE] Release Apache Hadoop 3.4.0 (RC2)

2024-02-23 Thread Steve Loughran
e ARM tar package, which should meet expectations. >> >> We also look forward to other members helping to verify. >> >> Best Regards, >> Shilun Fan. >> >> On Fri, Feb 16, 2024 at 12:22 AM Steve Loughran >> wrote: >> >> > >> &

Re: [VOTE] Release Apache Hadoop 3.4.0 (RC2)

2024-02-15 Thread Steve Loughran
On Mon, 12 Feb 2024 at 15:32, slfan1989 wrote: > > > Note, because the arm64 binaries are built separately on a different > platform and JVM, their jar files may not match those of the x86 > release -and therefore the maven artifacts. I don't think this is > an issue (the ASF actually releases so

Re: [VOTE] Release Apache Hadoop 3.4.0 (RC1)

2024-02-12 Thread Steve Loughran
so it'll have the s3a checksum and region stuff? that'd be wonderful! I've been assuming i'd have to push out a 3.4.1 release for that alone. On Sat, 10 Feb 2024 at 23:36, slfan1989 wrote: > We will end the voting for hadoop-3.4.0-RC1 and will open the voting for > hadoop-3.4.0-RC2 in 1-2 days.

Re: Fw:Re: [VOTE] Release Apache Hadoop 3.4.0 RC0

2024-01-15 Thread Steve Loughran
-1 I'm afraid, just due to staging/packaging issues. This took me a few goes to get right myself, so nothing unusual. Note I used my validator project which is set to retrieve binaries, check signatures, run maven builds against staged artifacts *and clean up any local copies first*and more. Thi

Re: [VOTE] Release Apache Hadoop 3.4.0 RC0

2024-01-11 Thread Steve Loughran
wonderful! I'll be testing over the weekend Meanwhile, new changes I'm putting in to trunk are tagged as fixed in 3.5.0 -correct? steve On Thu, 11 Jan 2024 at 05:15, slfan1989 wrote: > Hello all, > > We plan to release hadoop 3.4.0 based on hadoop trunk, which is the first > hadoop 3.4.0-RC v

Re: [DISCUSS][HDFS] Add rust binding for libhdfs

2024-01-01 Thread Steve Loughran
i hear owen o'malley has been learning rust, and as he left cloudera a year ago, he'll be missing github and JIRA On Thu, 21 Dec 2023 at 15:00, Ayush Saxena wrote: > It looks pretty challenging to me. Most of the committers aren't > technically equipped to review this code, so getting the in

Re: [VOTE] Hadoop 3.2.x EOL

2023-12-06 Thread Steve Loughran
+1 On Wed, 6 Dec 2023 at 04:09, Xiaoqiao He wrote: > Dear Hadoop devs, > > Given the feedback from the discussion thread [1], I'd like to start > an official thread for the community to vote on release line 3.2 EOL. > > It will include, > a. An official announcement informs no further regular Ha

Re: [DISCUSS] Make some release lines EOL

2023-12-05 Thread Steve Loughran
+1 for making 3.3 and 3.4 the maintained lines 3.2.x we should say -as it is true- that the age of the dependencies is such that it is transitively insecure. To fix those, people must upgrade. For 2.10.x, we should think about whether to cherrypick our own CVEs there, but not actually do any new

[jira] [Resolved] (HDFS-17249) Fix TestDFSUtil.testIsValidName() unit test failure

2023-11-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-17249. --- Fix Version/s: 3.4.0 Resolution: Fixed > Fix TestDFSUtil.testIsValidName() unit t

[jira] [Resolved] (HDFS-16791) Add getEnclosingRoot() API to filesystem interface and implementations

2023-11-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-16791. --- Resolution: Fixed > Add getEnclosingRoot() API to filesystem interface and implementati

Fwd: [jira] [Created] (HADOOP-18933) upgrade netty to 4.1.100 due to CVE

2023-10-16 Thread Steve Loughran
Can anyone from yarn or hdfs review this before i just upvote it hoping all will be well. thanks -- Forwarded message - From: PJ Fanning (Jira) Date: Wed, 11 Oct 2023 at 20:27 Subject: [jira] [Created] (HADOOP-18933) upgrade netty to 4.1.100 due to CVE To: PJ Fanning created HA

[jira] [Created] (HDFS-17224) TestRollingUpgrade.testDFSAdminRollingUpgradeCommands failing

2023-10-13 Thread Steve Loughran (Jira)
Steve Loughran created HDFS-17224: - Summary: TestRollingUpgrade.testDFSAdminRollingUpgradeCommands failing Key: HDFS-17224 URL: https://issues.apache.org/jira/browse/HDFS-17224 Project: Hadoop HDFS

Fwd: [jira] [Created] (HADOOP-18924) upgrade grpc jars to v1.53.0 due to CVEs

2023-10-09 Thread Steve Loughran
can we get some reviews from users of grpc/netty code here. otherwise i'd upvoting without knowing what will break. thanks -- Forwarded message - From: PJ Fanning (Jira) Date: Mon, 9 Oct 2023 at 11:46 Subject: [jira] [Created] (HADOOP-18924) upgrade grpc jars to v1.53.0 due to CVE

[jira] [Created] (HDFS-17202) TestDFSAdmin.testAllDatanodesReconfig assertion failing (again)

2023-09-19 Thread Steve Loughran (Jira)
Steve Loughran created HDFS-17202: - Summary: TestDFSAdmin.testAllDatanodesReconfig assertion failing (again) Key: HDFS-17202 URL: https://issues.apache.org/jira/browse/HDFS-17202 Project: Hadoop HDFS

Re: Signing releases using automated release infra

2023-07-20 Thread Steve Loughran
could be good. why not set it up for the third-party module first to see how well it works? On Tue, 18 Jul 2023 at 21:05, Ayush Saxena wrote: > Something we can explore as well!! > > -Ayush > > Begin forwarded message: > > > From: Volkan Yazıcı > > Date: 19 July 2023 at 1:24:49 AM IST > > To:

Re: [VOTE] Release Apache Hadoop 3.3.6 RC0

2023-06-21 Thread Steve Loughran
1. we should patch gcs, to ignore those new keys. 2. i may be able to validate RC1 this weekend On Fri, 16 Jun 2023 at 03:20, Wei-Chiu Chuang wrote: > Overall so far so good. > > hadoop-api-shim: > built, tested successfully. > > cloudstore: > built successfully. > > Spark: > built successfull

Re: [VOTE] Release Apache Hadoop 3.3.6 RC0

2023-06-15 Thread Steve Loughran
Which branch is -3.3.6 off? 3.3.5 or 3.3? I'm travelling for the next few days and unlikely to be able to test this; will do my best On Wed, 14 Jun 2023 at 07:27, Wei-Chiu Chuang wrote: > I am inviting anyone to try and vote on this release candidate. > > Note: > This is built off branch-3.3.6

Re: [DISCUSS] Hadoop 3.3.6 release planning

2023-05-05 Thread Steve Loughran
Wei-Chiu has suggested a minimal "things in 3.3.5 which were very broken, api change for ozone and any critical jar updates" so much lower risk/easier to qualify and ship. I need to get https://issues.apache.org/jira/browse/HADOOP-18724 in here; maybe look at a refresh of the "classic" jars (slf4

Re: [VOTE] Release Apache Hadoop 3.3.5 (RC3)

2023-03-31 Thread Steve Loughran
disable it. Will do it > tomorrow, if nobody objects. In case the one who configured it wants > to do it early, feel free to do so. > > We already have one for branch-3.3 which runs weekly which most > probably most of us don't follow :) > > -Ayush > > On Wed, 22 Mar 2

Re: [DISCUSS] Move HDFS specific APIs to FileSystem abstration

2023-03-29 Thread Steve Loughran
for your studious pursuit of this goal. > > Thanks, > Nick > > On Mon, Mar 27, 2023 at 4:58 AM Steve Loughran > > > wrote: > > > > > side issue, as i think about what bulk delete call would also keep > hbase > > > happy > > >

Re: [DISCUSS] hadoop branch-3.3+ going to java11 only

2023-03-28 Thread Steve Loughran
ongoing major dependency upgrades (log4j 1 to 2, jersey 1 > to 2 and junit 4 to 5) are blockers for java 11 compile + test stability. > > > On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran > > wrote: > > > Now that hadoop 3.3.5 is out, i want to propose something new >

[DISCUSS] hadoop branch-3.3+ going to java11 only

2023-03-28 Thread Steve Loughran
Now that hadoop 3.3.5 is out, i want to propose something new we switch branch-3.3 and trunk to being java11 only 1. java 11 has been out for years 2. oracle java 8 is no longer available under "premier support"; you can't really get upgrades https://www.oracle.com/java/technologies

Re: [DISCUSS] Move HDFS specific APIs to FileSystem abstration

2023-03-27 Thread Steve Loughran
outgrown checked exceptions On Fri, 24 Mar 2023 at 09:44, Steve Loughran wrote: > > > On Thu, 23 Mar 2023 at 10:07, Ayush Saxena wrote: > >> >> Second idea mentioned in the original mail is also similar to mentioned in >> the comment in the above ticket and is still

Re: [DISCUSS] Move HDFS specific APIs to FileSystem abstration

2023-03-24 Thread Steve Loughran
On Thu, 23 Mar 2023 at 10:07, Ayush Saxena wrote: > > Second idea mentioned in the original mail is also similar to mentioned in > the comment in the above ticket and is still quite acceptable, name can be > negotiated though, Add an interface to pull the relevant methods up in that > without tou

[ANNOUNCE] Apache Hadoop 3.3.5 release

2023-03-23 Thread Steve Loughran
On behalf of the Apache Hadoop Project Management Committee, I am pleased to announce the release of Apache Hadoop 3.3.5. This is a release of Apache Hadoop 3.3 line. Key changes include * A big update of dependencies to try and keep those reports of transitive CVEs under control -both genuin

Re: [VOTE] Release Apache Hadoop 3.3.5 (RC3)

2023-03-21 Thread Steve Loughran
ok, here's my summary, even though most of the binding voters forgot to declare they were on the PMC. +1 binding Steve Loughran Chris Nauroth Masatake Iwasaki Ayush Saxena Xiaoqiao He +1 non-binding Viraj Jasani 0 or -1 votes: none. Accordingly: the release is good! I will send the f

Re: [VOTE] Release Apache Hadoop 3.3.5 (RC3)

2023-03-18 Thread Steve Loughran
with scale profile looks good > > > On Wed, Mar 15, 2023 at 12:48 PM Steve Loughran > > wrote: > > > Apache Hadoop 3.3.5 > > > > Mukund and I have put together a release candidate (RC3) for Hadoop > 3.3.5. > > > > What we would like is for anyon

Re: [VOTE] Release Apache Hadoop 3.3.5 (RC3)

2023-03-17 Thread Steve Loughran
ypto.so ISA-L: false libhadoop was built without ISA-L support PMDK:false The native code was built without PMDK support. --- On Wed, 15 Mar 2023 at 19:47, Steve Loughran wrote: > > Apache Hadoop 3.3.5 > > Mukund and I have put together a release candidate (RC3) for Hadoop 3.3.5. >

Re: [DISCUSS] Move HDFS specific APIs to FileSystem abstration

2023-03-17 Thread Steve Loughran
1. I think a new interface would be good as FileContext could do the same thing 2. using PathCapabilities probes should still be mandatory as for FileContext it would depend on the back end 3. Whoever does this gets to specify what the API does and write the contract tests. Saying

[VOTE] Release Apache Hadoop 3.3.5 (RC3)

2023-03-15 Thread Steve Loughran
Apache Hadoop 3.3.5 Mukund and I have put together a release candidate (RC3) for Hadoop 3.3.5. What we would like is for anyone who can to verify the tarballs, especially anyone who can try the arm64 binaries as we want to include them too. The RC is available at: https://dist.apache.org/repos/d

Re: [VOTE] Release Apache Hadoop 3.3.5 (RC2)

2023-03-07 Thread Steve Loughran
does not exist (#5400) [ZanderXu < > zande...@apache.org > >] > * 44bf8aadedf 2023-03-03 HDFS-16832. [SBN READ] Follow-on to HDFS-16732. > Fix NPE when check the block location of empty directory (#5099) > [zhengchenyu ] > * 72f8c2a4888 (tag: release-3.3.5-RC2) 2023-02-25 HADOOP-18641

[jira] [Resolved] (HDFS-16934) org.apache.hadoop.hdfs.tools.TestDFSAdmin#testAllDatanodesReconfig regression

2023-03-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-16934. --- Fix Version/s: 3.4.0 3.3.5 Resolution: Fixed fixed; ran new test

Re: [VOTE] Release Apache Hadoop 3.3.5 (RC2)

2023-03-06 Thread Steve Loughran
imeout). > > > On Sat, Mar 4, 2023 at 4:29 AM Steve Loughran > > wrote: > > > On Sat, 4 Mar 2023 at 01:47, Erik Krogen wrote: > > > > > Thanks Steve. I see now that the branch cut was way back in October so > I > > > definitely understand your

Re: [VOTE] Release Apache Hadoop 3.3.5 (RC2)

2023-03-04 Thread Steve Loughran
>> - mvn clean install -DskipTests >> >> * Built tar from source (1.8.0_341): ok >> >> - mvn clean package -Pdist -DskipTests -Dtar >> -Dmaven.javadoc.skip=true >> >> >> >> * Built images using the tarball, installed and started all o

Re: [VOTE] Release Apache Hadoop 3.3.5 (RC2)

2023-03-03 Thread Steve Loughran
> * Ran Hbase (latest 2.5) tests against Hdfs, ran RowCounter Mapreduce job >> * Hdfs CRUD tests >> * MapReduce wordcount job >> >> * Ran S3A tests with scale profile against us-west-2: >> mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Dscale >> >> I

Re: [VOTE] Release Apache Hadoop 3.3.5 (RC2)

2023-03-02 Thread Steve Loughran
like to see about merging to branch-3.3.5 as well. > > Thanks for the consideration and sorry for not bringing this up in RC1 or > earlier. > > On Mon, Feb 27, 2023 at 9:59 AM Steve Loughran > > wrote: > > > Mukund and I have put together a release candidate (RC2)

[jira] [Resolved] (HDFS-16935) TestFsDatasetImpl.testReportBadBlocks brittle

2023-03-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-16935. --- Resolution: Fixed > TestFsDatasetImpl.testReportBadBlocks brit

[VOTE] Release Apache Hadoop 3.3.5 (RC2)

2023-02-27 Thread Steve Loughran
Mukund and I have put together a release candidate (RC2) for Hadoop 3.3.5. We need anyone who can to verify the source and binary artifacts, including those JARs staged on maven, the site documentation and the arm64 tar file. The RC is available at: https://dist.apache.org/repos/dist/dev/hadoop/h

Re: [VOTE] Release Apache Hadoop 3.3.5

2023-02-24 Thread Steve Loughran
without that file being up to date, but at least all the transitive stuff is correctly licensed. And i think we need to change the PR template to mention transitive updates in the license bit too if this goes in, I will do the rebuild on monday UK time On Thu, 23 Feb 2023 at 11:18, Steve Loughran

[jira] [Created] (HDFS-16935) TestFsDatasetImpl.testReportBadBlocks brittle

2023-02-24 Thread Steve Loughran (Jira)
Steve Loughran created HDFS-16935: - Summary: TestFsDatasetImpl.testReportBadBlocks brittle Key: HDFS-16935 URL: https://issues.apache.org/jira/browse/HDFS-16935 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-16934) org.apache.hadoop.hdfs.tools.TestDFSAdmin#testAllDatanodesReconfig regression

2023-02-24 Thread Steve Loughran (Jira)
Steve Loughran created HDFS-16934: - Summary: org.apache.hadoop.hdfs.tools.TestDFSAdmin#testAllDatanodesReconfig regression Key: HDFS-16934 URL: https://issues.apache.org/jira/browse/HDFS-16934

Re: [VOTE] Release Apache Hadoop 3.3.5

2023-02-23 Thread Steve Loughran
over time. Going to revert HADOOP-18590. Publish SBOM artifacts (#5281) On Thu, 23 Feb 2023 at 10:29, Steve Loughran wrote: > ok, let me cancel, update those jiras and kick off again. that will save > anyone else having to do their homework > > On Thu, 23 Feb 2023 at 08:56, Tak

Re: [VOTE] Release Apache Hadoop 3.3.5

2023-02-23 Thread Steve Loughran
03 > > - HDFS-16310 > > - HADOOP-18014 > > > > I corrected all the wrong fix versions just now. I'm not sure we should > > revote it since it only affects the changelog. > > > > - Takanobu > > > > 2023年2月21日(火) 22:43 Steve Loughran : > > >

[VOTE] Release Apache Hadoop 3.3.5

2023-02-21 Thread Steve Loughran
Apache Hadoop 3.3.5 Mukund and I have put together a release candidate (RC1) for Hadoop 3.3.5. What we would like is for anyone who can to verify the tarballs, especially anyone who can try the arm64 binaries as we want to include them too. The RC is available at: https://dist.apache.org/repos/d

[jira] [Resolved] (HDFS-16853) The UT TestLeaseRecovery2#testHardLeaseRecoveryAfterNameNodeRestart failed because HADOOP-18324

2023-02-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-16853. --- Fix Version/s: 3.3.5 Resolution: Fixed > The UT TestLeaseRecove

Re: [VOTE] Release Apache Hadoop 3.3.5

2023-01-16 Thread Steve Loughran
thanks pulling in a few of the recent changes which seem needed/important, now wondering about the javadocs. i will add a new probe for this in our automated release ant bulld so we can't cut a release without that https://github.com/steveloughran/validate-hadoop-client-artifacts On Mon, 2 Jan 2

[VOTE] Release Apache Hadoop 3.3.5

2022-12-21 Thread Steve Loughran
Mukund and I have put together a release candidate (RC0) for Hadoop 3.3.5. Given the time of year it's a bit unrealistic to run a 5 day vote and expect people to be able to test it thoroughly enough to make this the one we can ship. What we would like is for anyone who can to verify the tarballs,

exciting new content needed for the 3.3.5 index.md file

2022-12-01 Thread Steve Loughran
The first "smoke test" RC is about to be up for people to play with, we are just testing things here and getting that arm build done. Can I have some content for the index.html page describing what has changed? hadoop-project/src/site/markdown/index.md.vm I can (and will) speak highly of stuff I'

[jira] [Resolved] (HDFS-16795) Use secure XML parser utils in hdfs classes

2022-10-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-16795. --- Fix Version/s: 3.3.5 Resolution: Fixed > Use secure XML parser utils in hdfs clas

Re: [DISCUSS] Hadoop 3.3.5 release planning

2022-10-11 Thread Steve Loughran
tests needed tuning fixed now HADOOP-18401 > > > On Tue, Sep 27, 2022 at 9:35 AM Steve Loughran > > wrote: > > > Mukund has just created the new Hadoop release JIRA, > > https://issues.apache.org/jira/browse/HADOOP-18470, and is doing the > first > > build

HADOOP-18470 Release hadoop 3.3.5

2022-09-27 Thread Steve Loughran
Mukund has just created the new Hadoop release JIRA, https://issues.apache.org/jira/browse/HADOOP-18470, and is doing the first build/test before pushing it up. This is off branch-3.3, so it will have more significant changes than the 3.3.3 and 3.3.4 releases, which were just CVE/integration fixes.

the next hadoop 3.3.x release

2022-09-09 Thread Steve Loughran
Hi Mukund Thakur plans to fork off the next branch-3.3. release in 10 days time. last chance to get changes in. I'm away next week...try not to break the branch. Any testing you can do would be appreciated Steve

[jira] [Resolved] (HDFS-16755) TestQJMWithFaults.testUnresolvableHostName() can fail due to unexpected host resolution

2022-09-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-16755. --- Fix Version/s: 3.3.9 Resolution: Fixed > TestQJMWithFaults.testUnresolvableHostN

Re: Protobuf-java compatibilty question

2022-08-11 Thread Steve Loughran
I'm afraid not. protobuf maybe compatible at the wire level -the marshalled data is good-, but the generated classes in jars Will not link to any version other than that they were explicitly compiled with. this is why hadoop 3.3 has a private shaded copy in the hadoop-thirdparty JAR -how to stop th

[ANNOUNCE] Apache Hadoop 3.3.4 release

2022-08-08 Thread Steve Loughran
On behalf of the Apache Hadoop Project Management Committee, I am pleased to announce the release of Apache Hadoop 3.3.4. --- This is a release of Apache Hadoop 3.3 line. It contains a small number of security and critical integration fixes since 3.3.3. Users of Apache Hadoop 3.3.3 should upgrad

Re: [VOTE] Release Apache Hadoop 3.3.4

2022-08-08 Thread Steve Loughran
top > of it and all passed. Did some kills, stuff came back. > > I didn't spend time on unit tests but one set passed on a local rig here: > > [image: image.png] > Stack > > On Fri, Jul 29, 2022 at 11:48 AM Steve Loughran > wrote: > >> I have put together

Re: [VOTE] Release Apache Hadoop 3.3.4

2022-08-04 Thread Steve Loughran
The vote passed with the following result. Binding PMC members: +1 Chris Nauroth +1 Steve Loughran +1 Masatake Iwasaki Non binding votes: +1 Ashutosh Gupta Cheng Pan was worried about the new transient kotlin dependency. They are working on a PR there which we can target for the successor to

[jira] [Resolved] (HDFS-16711) Empty hadoop-client-api artifacts on maven central

2022-08-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-16711. --- Resolution: Won't Fix closing as "cantfix". sorry > Empty hadoop-clien

Re: [VOTE] Release Apache Hadoop 3.3.4

2022-08-03 Thread Steve Loughran
local projects https://github.com/steveloughran/validate-hadoop-client-artifacts all good as far as my test coverage goes, with these projects validating the staged dependencies. now, who else can review On Fri, 29 Jul 2022 at 19:47, Steve Loughran wrote: > > > I have put together

[VOTE] Release Apache Hadoop 3.3.4

2022-07-29 Thread Steve Loughran
I have put together a release candidate (RC1) for Hadoop 3.3.4 The RC is available at: https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/ The git tag is release-3.3.4-RC1, commit a585a73c3e0 The maven artifacts are staged at https://repository.apache.org/content/repositories/orgapac

any HDFS blockers for a 3.3.x feature release?

2022-07-28 Thread Steve Loughran
Is there any WiP which people consider a blocker for a 3.3.x feature release to follow the 3.3.4 security/critical integration issues release? I want to split off the fork for that next release off branch-3.3 and start stabilising. are there things people working on which are nearly ready?

[jira] [Resolved] (HDFS-9475) execution of org.apache.hadoop.hdfs.net.TcpPeerServer.close() causes timeout on Hadoop-2.6.0 with IBM-JDK-1.8

2022-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-9475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-9475. -- Resolution: Won't Fix > execution of org.apache.hadoop.hdfs.net.TcpPeerServer.close(

Re: [VOTE] Release Apache Hadoop 3.3.4

2022-07-26 Thread Steve Loughran
u, 21 Jul 2022 at 19:07, Steve Loughran wrote: > > I have put together a release candidate (RC0) for Hadoop 3.3.4 > > The RC is available at: > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC0/ > > The git tag is release-3.3.4-RC0, commit c679bc76d26 > > T

[jira] [Created] (HDFS-16679) StandbyState link

2022-07-22 Thread Steve Loughran (Jira)
Steve Loughran created HDFS-16679: - Summary: StandbyState link Key: HDFS-16679 URL: https://issues.apache.org/jira/browse/HDFS-16679 Project: Hadoop HDFS Issue Type: Bug Reporter

[VOTE] Release Apache Hadoop 3.3.4

2022-07-21 Thread Steve Loughran
I have put together a release candidate (RC0) for Hadoop 3.3.4 The RC is available at: https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC0/ The git tag is release-3.3.4-RC0, commit c679bc76d26 The maven artifacts are staged at https://repository.apache.org/content/repositories/orgapac

[jira] [Resolved] (HDFS-16651) io.netty:netty CVE-2019-20444, CVE-2019-20445

2022-07-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-16651. --- Resolution: Duplicate there's some outstanding netty upgrade jiras; remember to check b

Re: Hadoop 3.3.4 release process underway

2022-06-22 Thread Steve Loughran
n, 20 Jun 2022 at 19:02, Steve Loughran wrote: > I'm setting things up for a new release > > https://issues.apache.org/jira/browse/HADOOP-18305 > > absolute minimum of fixes. as well as some related to ZK lockdown, i would > like to include > > https://issues.apa

Hadoop 3.3.4 release process underway

2022-06-20 Thread Steve Loughran
I'm setting things up for a new release https://issues.apache.org/jira/browse/HADOOP-18305 absolute minimum of fixes. as well as some related to ZK lockdown, i would like to include https://issues.apache.org/jira/browse/HADOOP-18303 remove shading exclusion of javax.ws.rs-api from hadoop-client-

[DISCUSS] Filesystem API shim library to assist applications still targeting previous hadoop releases.

2022-06-08 Thread Steve Loughran
I've just created an initial project "fs-api-shim" to provide controlled access to the hadoop 3.3.3+ filesystem API calls on hadoop 3.2.0+ releases https://github.com/steveloughran/fs-api-shim The goal here is to make it possible for core file format libraries (Parquet, Avro, ORC, Arrow etc) and o

[DISCUSS] Forthcoming Hadoop releases

2022-06-08 Thread Steve Loughran
I want to start a quick discussion on a plan for hadoop releases this summer. I am willing to do the release manager work. Mukund and Mehakmeet have have already volunteered to help even if they don't know that yet. I've got two goals 1. minor followup to 3.3.3 2. feature release of new stu

[jira] [Resolved] (HDFS-16563) Namenode WebUI prints sensitive information on Token Expiry

2022-06-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-16563. --- Resolution: Fixed > Namenode WebUI prints sensitive information on Token Exp

[jira] [Reopened] (HDFS-16563) Namenode WebUI prints sensitive information on Token Expiry

2022-06-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HDFS-16563: --- > Namenode WebUI prints sensitive information on Token Exp

Re: [VOTE] Release Apache Hadoop 2.10.2 - RC0

2022-05-30 Thread Steve Loughran
+1 binding I've extended my validator project https://github.com/steveloughran/validate-hadoop-client-artifacts it now * fetches KEYS * fetches an RC from a remote url * validates signing * untars and tries to build source * untars binary release and tries some native commands the source build f

[ANNOUNCE] Apache Hadoop 3.3.3 release

2022-05-18 Thread Steve Loughran
On behalf of the Apache Hadoop Project Management Committee, I'm pleased to announce that Hadoop 3.3.3 has been released: https://hadoop.apache.org/release/3.3.3.html This is the third stable release of the Apache Hadoop 3.3 line. It contains 23 bug fixes, improvements and enhancements since 3.3

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

2022-05-17 Thread Steve Loughran
c. > > > > > > > > Thanks Steve for your work. > > > > > > > > - He Xiaoqiao > > > > > > > > On Mon, May 16, 2022 at 4:25 AM Viraj Jasani > > wrote: > > > > > > > > > > +1 (non-bind

[VOTE] Release Apache Hadoop 3.3.3 (RC1)

2022-05-11 Thread Steve Loughran
I have put together a release candidate (RC1) for Hadoop 3.3.3 The RC is available at: https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/ The git tag is release-3.3.3-RC1, commit d37586cbda3 The maven artifacts are staged at https://repository.apache.org/content/repositories/orgapachehadoo

Re: [VOTE] Release Apache Hadoop 3.3.3

2022-05-10 Thread Steve Loughran
version in staging with the files; still don't know how the previous delete lacked them, I will do more testing tomorrow before putting up the next RC. which will be off the same git commit as before, just a rebuild, repackage and republish. -Steve On Tue, 10 May 2022 at 00:18, Steve Lou

Re: [VOTE] Release Apache Hadoop 3.3.3

2022-05-09 Thread Steve Loughran
staged-releases >https://repository.apache.org/content/repositories/staging/ > > > true > > > true > > > > to the `` section in your Maven pom.xml file. > > On Mon, May 9, 2022 at 8:52 AM Steve

Re: [VOTE] Release Apache Hadoop 3.3.3

2022-05-09 Thread Steve Loughran
) > >>>>>> > >>>>>> * Signature: ok > >>>>>> * Checksum : passed > >>>>>> * Rat check (1.8.0_191): passed > >>>>>>- mvn clean apache-rat:check > >>>>>> * Built from

Re: [DISCUSS] Enabling all platform builds in CI for all Hadoop PRs

2022-05-09 Thread Steve Loughran
gt; >> to compensate for this time. > >> > >> For now lets just integrate it our nightly builds only and circle back > >> again here in future if the need be. > >> > >> -Ayush > >> > >>> On Fri, 6 May 2022 at 20:44, Wei-Chiu C

Re: [VOTE] Release Apache Hadoop 3.3.3

2022-05-08 Thread Steve Loughran
gt; - mvn clean install -DskipTests > > >>>> - mvn -fae --no-transfer-progress -DskipTests > > -Dmaven.javadoc.skip=true > > >>>> -Pnative -Drequire.openssl -Drequire.snappy -Drequire.valgrind > > >>>> -Drequire.zstd -Dreq

Re: [DISCUSS] Enabling all platform builds in CI for all Hadoop PRs

2022-05-06 Thread Steve Loughran
I'm not enthusiastic here as it not only makes the builds slower, it reduces the #of builds we can through a day one thing I am wondering is could we remove java8 support on some branches? make branch 3.3.2.x (i.e the 3.3.3 release) the last java 8 build, and this summers branch-3.3 release (whic

[VOTE] Release Apache Hadoop 3.3.3

2022-05-03 Thread Steve Loughran
I have put together a release candidate (rc0) for Hadoop 3.3.3 The RC is available at: https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/ The git tag is release-3.3.3-RC0, commit d37586cbda3 The maven artifacts are staged at https://repository.apache.org/content/repositories/orgapachehadoo

[jira] [Resolved] (HDFS-14478) Add libhdfs APIs for openFile

2022-04-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-14478. --- Fix Version/s: 3.3.4 Resolution: Fixed > Add libhdfs APIs for openF

[jira] [Resolved] (HDFS-16501) Print the exception when reporting a bad block

2022-04-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-16501. --- Resolution: Fixed > Print the exception when reporting a bad bl

[jira] [Resolved] (HDFS-16437) ReverseXML processor doesn't accept XML files without the SnapshotDiffSection.

2022-04-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-16437. --- Resolution: Fixed > ReverseXML processor doesn't accept XML files wit

[jira] [Resolved] (HDFS-16355) Improve the description of dfs.block.scanner.volume.bytes.per.second

2022-04-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-16355. --- Fix Version/s: 3.3.3 (was: 3.4.0) (was: 3.3.4

[jira] [Resolved] (HDFS-11041) Unable to unregister FsDatasetState MBean if DataNode is shutdown twice

2022-04-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-11041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-11041. --- Fix Version/s: 3.3.3 (was: 3.4.0) (was: 3.3.4

[jira] [Resolved] (HDFS-16428) Source path with storagePolicy cause wrong typeConsumed while rename

2022-04-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-16428. --- Fix Version/s: 3.3.3 (was: 3.4.0) (was: 3.3.4

[jira] [Resolved] (HDFS-16507) [SBN read] Avoid purging edit log which is in progress

2022-04-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-16507. --- Fix Version/s: 3.3.3 Resolution: Fixed FIxed in 3.3.3; updating fix versions as

  1   2   3   4   5   6   >