Re: [ANNOUNCE] Eric Badger is now a committer!

2019-03-05 Thread Jim Brennan
Congratulations Eric! On Tue, Mar 5, 2019 at 11:20 AM Eric Payne wrote: > It is my pleasure to announce that Eric Badger has accepted an invitation > to become a Hadoop Core committer. > > Congratulations, Eric! This is well-deserved! > > -Eric Payne >

Re: [DISCUSS] Docker build process

2019-03-19 Thread Jim Brennan
I agree with Steve and Marton. I am ok with having the docker build as an option, but I don't want it to be the default. Jim On Tue, Mar 19, 2019 at 12:19 PM Eric Yang wrote: > Hi Marton, > > Thank you for your input. I agree with most of what you said with a few > exceptions. Security fix

Re: [VOTE] Merge YARN-8200 to branch-2 and branch-3.0

2019-08-26 Thread Jim Brennan
+1 (non-binding). I have built branch-2 with the latest YARN-8200 patch (YARN-8200-branch-2.003.patch). I ran all of the NM/RM tests and ran a few test jobs on a one-node cluster with default settings. On Mon, Aug 26, 2019 at 3:51 PM Oliver Hu wrote: > +1 (non-binding) > > We have used this pa

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

2019-10-25 Thread Jim Brennan
+1 (non-binding) on RC1 I built from source on Mac and RHEL7, ran hdfs, nodemanager, and resourcemanager unit tests, and set up a one-node cluster and ran some test jobs (pi and sleep). - Jim On Tue, Oct 22, 2019 at 4:55 PM Jonathan Hung wrote: > Hi folks, > > This is the second release candida

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2019-12-31 Thread Jim Brennan
It looks like QBT tests are still being run on branch-2 ( https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/), and they are not very helpful at this point. Can we change the QBT tests to run against branch-2.10 instead? Jim On Mon, Dec 23, 2019 at 7:44 PM Akira

Re: [E] Re: [DISCUSS] Hadoop 2.10.1 release

2020-09-02 Thread Jim Brennan
Thanks Masatake Iwasaki! I am willing to help out with Hadoop 2.10.1 release. Jim Brennan On Tue, Sep 1, 2020 at 2:13 AM Masatake Iwasaki wrote: > Thanks, Mingliang Liu. > > I volunteer to take the RM role then. > I will appreciate advice from who have the experience. > >

Re: [E] [VOTE] Release Apache Hadoop 2.10.1 (RC0)

2020-09-16 Thread Jim Brennan
Thanks for your work on this Masatake! I am +1 (non-binding) on this 2.10.1 release. I built from source and ran hdfs, resourcemanager, and nodemanager unit tests. I set up a one-node-cluster and ran some example jobs (pi, sleep). I tested NM/RM recovery by killing NM/RM during jobs and verifying

Re: [E] Cost Based FairCallQueue latency issue

2020-11-05 Thread Jim Brennan
Note that I have a Jira up right now for a bug that Daryn found while testing FCQ internally. Not sure if it is relevant to what you are seeing. https://issues.apache.org/jira/browse/HADOOP-17342 Jim On Thu, Nov 5, 2020 at 11:43 AM Fengnan Li wrote: > Thanks for the response Daryn! > > > > I ag

Re: [E] Re: [VOTE] Release Apache Hadoop 3.2.2 - RC4

2020-12-21 Thread Jim Brennan
I put up a patch for https://issues.apache.org/jira/browse/YARN-10540. Thanks for bringing it to my attention. Jim On Mon, Dec 21, 2020 at 10:36 AM Sunil Govindan wrote: > I had some offline talks with a few folks. > This issue is happening only in Mac, hence ideally it does not cause much > of

Re: [E] Re: Java 8 Lambdas

2021-04-29 Thread Jim Brennan
I just think that we should be cognizant of changes (particularly bug fixes), that will need to be ported to branch-2.10. Since it is still on Java7, anytime you use a lambda in code on trunk, we need to change it for branch-2.10. While not difficult, this is extra work and it increases the diff

Re: [E] [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Jim Brennan
+1 On Thu, Jun 3, 2021 at 1:14 AM Akira Ajisaka wrote: > Dear Hadoop developers, > > Given the feedback from the discussion thread [1], I'd like to start > an official vote > thread for the community to vote and start the 3.1 EOL process. > > What this entails: > > (1) an official announcement

[jira] [Created] (HDFS-14893) TestDelegationTokensWithHA.testObserverReadProxyProviderWithDT failing on branch-2

2019-10-04 Thread Jim Brennan (Jira)
Jim Brennan created HDFS-14893: -- Summary: TestDelegationTokensWithHA.testObserverReadProxyProviderWithDT failing on branch-2 Key: HDFS-14893 URL: https://issues.apache.org/jira/browse/HDFS-14893

[jira] [Created] (HDFS-14958) TestBalancerWithNodeGroup is not using NetworkTopologyWithNodeGroup

2019-11-06 Thread Jim Brennan (Jira)
Jim Brennan created HDFS-14958: -- Summary: TestBalancerWithNodeGroup is not using NetworkTopologyWithNodeGroup Key: HDFS-14958 URL: https://issues.apache.org/jira/browse/HDFS-14958 Project: Hadoop HDFS

[jira] [Created] (HDFS-14960) TesteBalancerWithNodeGroup should not succeed with DFSNetworkTopology

2019-11-06 Thread Jim Brennan (Jira)
Jim Brennan created HDFS-14960: -- Summary: TesteBalancerWithNodeGroup should not succeed with DFSNetworkTopology Key: HDFS-14960 URL: https://issues.apache.org/jira/browse/HDFS-14960 Project: Hadoop HDFS

[jira] [Reopened] (HDFS-13339) Volume reference can't be released and may lead to deadlock when DataXceiver does a check volume

2020-01-14 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan reopened HDFS-13339: Re-opening issue so I can put up a patch for branch-2.10. > Volume reference can't be rele

[jira] [Created] (HDFS-15125) Pull back fixes for DataNodeVolume* unit tests

2020-01-15 Thread Jim Brennan (Jira)
Jim Brennan created HDFS-15125: -- Summary: Pull back fixes for DataNodeVolume* unit tests Key: HDFS-15125 URL: https://issues.apache.org/jira/browse/HDFS-15125 Project: Hadoop HDFS Issue Type

[jira] [Reopened] (HDFS-10499) TestNameNodeMetadataConsistency#testGenerationStampInFuture Fails Intermittently

2020-03-13 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HDFS-10499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan reopened HDFS-10499: Re-opening for branch-2.10 patch supplied by [~ahussein] > TestNameNodeMetadataConsiste

[jira] [Reopened] (HDFS-11396) TestNameNodeMetadataConsistency#testGenerationStampInFuture timed out

2020-03-13 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HDFS-11396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan reopened HDFS-11396: Reopening to submit patch for branch-2.10. > TestNameNodeMetadataConsiste

[jira] [Reopened] (HDFS-12453) TestDataNodeHotSwapVolumes fails in trunk Jenkins runs

2020-06-11 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HDFS-12453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan reopened HDFS-12453: Re-opening so I can submit patch for branch-2.10. > TestDataNodeHotSwapVolumes fails in trunk Jenk

[jira] [Resolved] (HDFS-15694) Avoid calling UpdateHeartBeatState inside DataNodeDescriptor

2020-12-01 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan resolved HDFS-15694. Fix Version/s: 3.2.3 3.1.5 3.4.0 3.3.1

[jira] [Resolved] (HDFS-15695) NN should not let the balancer run in safemode

2020-12-02 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan resolved HDFS-15695. Fix Version/s: 3.2.3 3.1.5 3.4.0 3.3.1

[jira] [Resolved] (HDFS-15703) Don't generate edits for set operations that are no-op

2020-12-03 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan resolved HDFS-15703. Fix Version/s: 3.2.3 3.1.5 3.4.0 3.3.1

[jira] [Resolved] (HDFS-15707) NNTop counts don't add up as expected

2020-12-07 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan resolved HDFS-15707. Fix Version/s: 3.2.3 3.1.5 3.4.0 3.3.1

[jira] [Resolved] (HDFS-15716) TestUpgradeDomainBlockPlacementPolicy flaky

2020-12-08 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan resolved HDFS-15716. Fix Version/s: 3.2.3 2.10.2 3.1.5 3.4.0

[jira] [Resolved] (HDFS-15711) Add Metrics to HttpFS Server

2020-12-10 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan resolved HDFS-15711. Fix Version/s: 3.2.3 3.1.5 3.4.0 3.3.1

[jira] [Resolved] (HDFS-15717) Improve fsck logging

2020-12-11 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan resolved HDFS-15717. Fix Version/s: 3.4.0 3.3.1 3.2.2 Resolution: Fixed

[jira] [Resolved] (HDFS-15718) Optimize hdfs token op writeLock

2020-12-11 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan resolved HDFS-15718. Resolution: Later Closing this.  This change is part of a larger feature that we are not ready to

[jira] [Resolved] (HDFS-15704) Mitigate lease monitor's rapid infinite loop

2020-12-17 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan resolved HDFS-15704. Fix Version/s: 3.2.3 3.1.5 3.4.0 3.3.1

[jira] [Reopened] (HDFS-10498) Intermittent test failure org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength.testSnapshotfileLength

2021-01-25 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HDFS-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan reopened HDFS-10498: Assignee: Jim Brennan (was: Ahmed Hussein) We did end up fixing this internally, I will put up

[jira] [Created] (HDFS-15789) Lease renewal does not require namesystem lock

2021-01-25 Thread Jim Brennan (Jira)
Jim Brennan created HDFS-15789: -- Summary: Lease renewal does not require namesystem lock Key: HDFS-15789 URL: https://issues.apache.org/jira/browse/HDFS-15789 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-15813) DataStreamer: keep sending heartbeat packets while streaming

2021-02-02 Thread Jim Brennan (Jira)
Jim Brennan created HDFS-15813: -- Summary: DataStreamer: keep sending heartbeat packets while streaming Key: HDFS-15813 URL: https://issues.apache.org/jira/browse/HDFS-15813 Project: Hadoop HDFS

[jira] [Created] (HDFS-15823) TestReconstructStripedFileWithRandomECPolicy failures

2021-02-05 Thread Jim Brennan (Jira)
Jim Brennan created HDFS-15823: -- Summary: TestReconstructStripedFileWithRandomECPolicy failures Key: HDFS-15823 URL: https://issues.apache.org/jira/browse/HDFS-15823 Project: Hadoop HDFS Issue

[jira] [Resolved] (HDFS-16042) DatanodeAdminMonitor scan should be delay based

2021-06-11 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan resolved HDFS-16042. Fix Version/s: 3.4.0 Resolution: Fixed Thanks for the contribution [~ahussein]! [~kihwal