Re: [ANNOUNCE] New Hadoop Committer - Simbarashe Dzinamarira

Congratulations Simba! Thanks for the great work you've done on making HDFS more scalable! On Mon, Oct 2, 2023 at 4:31 PM Iñigo Goiri wrote: > I am pleased to announce that Simbarashe Dzinamarira has been elected as a > committer on the Apache Hadoop project. > We appreciate all of Simbarashe's

Re: [VOTE] Release Apache Hadoop 3.3.5 (RC2)

le to get completed > > within timeout). > > > > > > On Sat, Mar 4, 2023 at 4:29 AM Steve Loughran > > > > > wrote: > > > > > On Sat, 4 Mar 2023 at 01:47, Erik Krogen wrote: > > > > > > > Thanks Steve. I see now tha

Re: [VOTE] Release Apache Hadoop 3.3.5 (RC2)

Thanks Steve. I see now that the branch cut was way back in October so I definitely understand your frustration here! This made me realize that HDFS-16832 , which resolves a very similar issue as the aforementioned HDFS-16923, is also missing from

Re: [VOTE] Release Apache Hadoop 3.3.5 (RC2)

Hi folks, apologies for being late to the release conversation, but I think we need to get HDFS-16923 into 3.3.5. HDFS-16732 , which also went into 3.3.5, introduced an issue whereby Observer

[jira] [Resolved] (HDFS-16923) The getListing RPC will throw NPE if the path does not exist

[ https://issues.apache.org/jira/browse/HDFS-16923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-16923. Resolution: Fixed > The getListing RPC will throw NPE if the path does not ex

[jira] [Resolved] (HDFS-16764) ObserverNamenode handles addBlock rpc and throws a FileNotFoundException

[ https://issues.apache.org/jira/browse/HDFS-16764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-16764. Resolution: Fixed > ObserverNamenode handles addBlock rpc and throws a FileNotFoundExcept

[jira] [Resolved] (HDFS-16872) Fix log throttling by declaring LogThrottlingHelper as static members

[ https://issues.apache.org/jira/browse/HDFS-16872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-16872. Resolution: Fixed > Fix log throttling by declaring LogThrottlingHelper as static memb

[jira] [Resolved] (HDFS-16689) Standby NameNode crashes when transitioning to Active with in-progress tailer

[ https://issues.apache.org/jira/browse/HDFS-16689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-16689. Resolution: Fixed > Standby NameNode crashes when transitioning to Active with in-progress tai

[jira] [Resolved] (HDFS-16852) Register the shutdown hook only when not in shutdown for KeyProviderCache constructor

[ https://issues.apache.org/jira/browse/HDFS-16852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-16852. Resolution: Fixed > Register the shutdown hook only when not in shutdown for KeyProviderCa

[jira] [Resolved] (HDFS-16550) [SBN read] Improper cache-size for journal node may cause cluster crash

[ https://issues.apache.org/jira/browse/HDFS-16550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-16550. Fix Version/s: 3.4.0 Resolution: Fixed > [SBN read] Improper cache-size for journal n

[jira] [Resolved] (HDFS-16547) [SBN read] Namenode in safe mode should not be transfered to observer state

[ https://issues.apache.org/jira/browse/HDFS-16547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-16547. Resolution: Fixed > [SBN read] Namenode in safe mode should not be transfered to observer st

[jira] [Resolved] (HDFS-16832) [SBN READ] Fix NPE when check the block location of empty directory

[ https://issues.apache.org/jira/browse/HDFS-16832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-16832. Resolution: Fixed > [SBN READ] Fix NPE when check the block location of empty direct

[jira] [Resolved] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId

[ https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-16659. Fix Version/s: 3.4.0 Resolution: Fixed > JournalNode should throw NewerTxnIdExcept

[jira] [Resolved] (HDFS-16732) [SBN READ] Avoid get location from observer when the block report is delayed.

[ https://issues.apache.org/jira/browse/HDFS-16732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-16732. Fix Version/s: 3.4.0 3.3.9 Resolution: Fixed Merged PR 4756 to trunk

[jira] [Resolved] (HDFS-16181) [SBN Read] Fix metric of RpcRequestCacheMissAmount can't display when tailEditLog form JN

[ https://issues.apache.org/jira/browse/HDFS-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-16181. Fix Version/s: 3.1.5 3.2.4 3.3.2 2.10.2

[jira] [Resolved] (HDFS-16233) Do not use exception handler to implement copy-on-write for EnumCounters

[ https://issues.apache.org/jira/browse/HDFS-16233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-16233. Resolution: Fixed > Do not use exception handler to implement copy-on-write for EnumCount

[jira] [Created] (HDFS-15032) Balancer crashes when it fails to contact an NN via ObserverReadProxyProvider

Erik Krogen created HDFS-15032: -- Summary: Balancer crashes when it fails to contact an NN via ObserverReadProxyProvider Key: HDFS-15032 URL: https://issues.apache.org/jira/browse/HDFS-15032 Project

Re: [DISCUSS] Making 2.10 the last minor 2.x release

I'm in support of this. The current scheme is confusing, and as you mentioned, is making the backport strategy less clear. It reminds me of the branch-2.8 vs. branch-2 (destined for 2.9) days when various fixes would make it into one or the other. One other action item would be to do a quick

[jira] [Created] (HDFS-14979) [Observer Node] Balancer should submit getBlocks to Observer Node when possible

Erik Krogen created HDFS-14979: -- Summary: [Observer Node] Balancer should submit getBlocks to Observer Node when possible Key: HDFS-14979 URL: https://issues.apache.org/jira/browse/HDFS-14979 Project

[jira] [Created] (HDFS-14973) Balancer getBlocks RPC dispersal does not function properly

Erik Krogen created HDFS-14973: -- Summary: Balancer getBlocks RPC dispersal does not function properly Key: HDFS-14973 URL: https://issues.apache.org/jira/browse/HDFS-14973 Project: Hadoop HDFS

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

Sounds good to me, thank you for addressing the concerns Jonathan. +1 from me. From: Masatake Iwasaki Sent: Monday, October 28, 2019 7:52 AM To: Jonathan Hung ; Hdfs-dev ; mapreduce-dev ; yarn-dev ; Hadoop Common Subject: Re: [VOTE] Release Apache Hadoop

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

Thanks for putting this together, Jonathan! I noticed that the RELEASENOTES.md makes no mention of any of the major features you mentioned in your email about the RC. Is this expected? I guess it is caused by the lack of a release note on the JIRAs for those features. I also noticed that

[jira] [Resolved] (HDFS-14245) Class cast error in GetGroups with ObserverReadProxyProvider

[ https://issues.apache.org/jira/browse/HDFS-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-14245. Resolution: Fixed > Class cast error in GetGroups with ObserverReadProxyProvi

[jira] [Reopened] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reopened HDFS-14162: Re-opening for backport to older branches which should have been done from the start. > Balan

[jira] [Reopened] (HDFS-14245) Class cast error in GetGroups with ObserverReadProxyProvider

[ https://issues.apache.org/jira/browse/HDFS-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reopened HDFS-14245: Re-opening for backport to other branches, which should have been done from the start. > Class c

[jira] [Created] (HDFS-14829) [Dynamometer] Update TestDynamometerInfra to be Hadoop 3.2+ compatible

Erik Krogen created HDFS-14829: -- Summary: [Dynamometer] Update TestDynamometerInfra to be Hadoop 3.2+ compatible Key: HDFS-14829 URL: https://issues.apache.org/jira/browse/HDFS-14829 Project: Hadoop

Re: [VOTE] Mark 2.6, 2.7, 3.0 release lines EOL

+1 thanks for initiating an official vote Wangda! From: Jonathan Eagles Sent: Wednesday, August 21, 2019 8:22 AM To: Sunil Govindan Cc: Wangda Tan ; yarn-dev ; Hdfs-dev ; mapreduce-dev ; Hadoop Common Subject: Re: [VOTE] Mark 2.6, 2.7, 3.0 release lines EOL

[jira] [Created] (HDFS-14667) Backport [HDFS-14403] "Cost-based FairCallQueue" to branch-2

Erik Krogen created HDFS-14667: -- Summary: Backport [HDFS-14403] "Cost-based FairCallQueue" to branch-2 Key: HDFS-14667 URL: https://issues.apache.org/jira/browse/HDFS-14667 Project: H

[jira] [Created] (HDFS-14643) [Dynamometer] Merge extra commits from GitHub to Hadoop

Erik Krogen created HDFS-14643: -- Summary: [Dynamometer] Merge extra commits from GitHub to Hadoop Key: HDFS-14643 URL: https://issues.apache.org/jira/browse/HDFS-14643 Project: Hadoop HDFS

[jira] [Created] (HDFS-14640) [Dynamometer] Fix TestDynamometerInfra failures

Erik Krogen created HDFS-14640: -- Summary: [Dynamometer] Fix TestDynamometerInfra failures Key: HDFS-14640 URL: https://issues.apache.org/jira/browse/HDFS-14640 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-14639) [Dynamometer] Unnecessary duplicate bin directory appears in dist layout

Erik Krogen created HDFS-14639: -- Summary: [Dynamometer] Unnecessary duplicate bin directory appears in dist layout Key: HDFS-14639 URL: https://issues.apache.org/jira/browse/HDFS-14639 Project: Hadoop

[jira] [Created] (HDFS-14638) [Dynamometer] Fix scripts to refer to current build structure

Erik Krogen created HDFS-14638: -- Summary: [Dynamometer] Fix scripts to refer to current build structure Key: HDFS-14638 URL: https://issues.apache.org/jira/browse/HDFS-14638 Project: Hadoop HDFS

[jira] [Created] (HDFS-14539) Remove Dynamometer's reliance on the tar utility

Erik Krogen created HDFS-14539: -- Summary: Remove Dynamometer's reliance on the tar utility Key: HDFS-14539 URL: https://issues.apache.org/jira/browse/HDFS-14539 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-14500) NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished

[ https://issues.apache.org/jira/browse/HDFS-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-14500. Resolution: Fixed > NameNode StartupProgress continues to report edit log segments af

[jira] [Reopened] (HDFS-14500) NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished

[ https://issues.apache.org/jira/browse/HDFS-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reopened HDFS-14500: > NameNode StartupProgress continues to report edit log segments after the > LOADING_EDITS

[jira] [Created] (HDFS-14500) NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished

Erik Krogen created HDFS-14500: -- Summary: NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished Key: HDFS-14500 URL: https://issues.apache.org/jira/browse/HDFS

[jira] [Created] (HDFS-14462) WebHDFS throws "Error writing request body to server" instead of NSQuotaExceededException

Erik Krogen created HDFS-14462: -- Summary: WebHDFS throws "Error writing request body to server" instead of NSQuotaExceededException Key: HDFS-14462 URL: https://issues.apache.org/jira/browse/

[jira] [Created] (HDFS-14442) Disagreement between HAUtil.getAddressOfActive and RpcInvocationHandler.getConnectionId

Erik Krogen created HDFS-14442: -- Summary: Disagreement between HAUtil.getAddressOfActive and RpcInvocationHandler.getConnectionId Key: HDFS-14442 URL: https://issues.apache.org/jira/browse/HDFS-14442

[jira] [Created] (HDFS-14435) ObserverReadProxyProvider is unable to properly fetch HAState from Standby NNs

Erik Krogen created HDFS-14435: -- Summary: ObserverReadProxyProvider is unable to properly fetch HAState from Standby NNs Key: HDFS-14435 URL: https://issues.apache.org/jira/browse/HDFS-14435 Project

[jira] [Created] (HDFS-14413) HA Support for Dynamometer

Erik Krogen created HDFS-14413: -- Summary: HA Support for Dynamometer Key: HDFS-14413 URL: https://issues.apache.org/jira/browse/HDFS-14413 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-14412) Enable Dynamometer to use the local build of Hadoop by default

Erik Krogen created HDFS-14412: -- Summary: Enable Dynamometer to use the local build of Hadoop by default Key: HDFS-14412 URL: https://issues.apache.org/jira/browse/HDFS-14412 Project: Hadoop HDFS

[jira] [Created] (HDFS-14411) Combine Dynamometer's SimulatedDataNodes into DataNodeCluster

Erik Krogen created HDFS-14411: -- Summary: Combine Dynamometer's SimulatedDataNodes into DataNodeCluster Key: HDFS-14411 URL: https://issues.apache.org/jira/browse/HDFS-14411 Project: Hadoop HDFS

[jira] [Created] (HDFS-14410) Make Dynamometer documentation properly compile onto the Hadoop site

Erik Krogen created HDFS-14410: -- Summary: Make Dynamometer documentation properly compile onto the Hadoop site Key: HDFS-14410 URL: https://issues.apache.org/jira/browse/HDFS-14410 Project: Hadoop HDFS

[jira] [Created] (HDFS-14409) Improve Dynamometer test suite

Erik Krogen created HDFS-14409: -- Summary: Improve Dynamometer test suite Key: HDFS-14409 URL: https://issues.apache.org/jira/browse/HDFS-14409 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-14370) Edit log tailing fast-path should allow for backoff

Erik Krogen created HDFS-14370: -- Summary: Edit log tailing fast-path should allow for backoff Key: HDFS-14370 URL: https://issues.apache.org/jira/browse/HDFS-14370 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-14349) Edit log may be rolled more frequently than necessary with multiple Standby nodes

Erik Krogen created HDFS-14349: -- Summary: Edit log may be rolled more frequently than necessary with multiple Standby nodes Key: HDFS-14349 URL: https://issues.apache.org/jira/browse/HDFS-14349 Project

[jira] [Created] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

Erik Krogen created HDFS-14279: -- Summary: [SBN Read] Race condition in ObserverReadProxyProvider Key: HDFS-14279 URL: https://issues.apache.org/jira/browse/HDFS-14279 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

Erik Krogen created HDFS-14211: -- Summary: [Consistent Observer Reads] Allow for configurable "always msync" mode Key: HDFS-14211 URL: https://issues.apache.org/jira/browse/HDFS-14211 Proje

Hadoop Contributor Meetup Hosted by LinkedIn

Hello fellow Hadoopers! LinkedIn is pleased to be hosting an all-day meetup event for Hadoop contributors and community members to gather together to share information and insights on the present, past and future of the Hadoop software we all know and love. The day will begin with a series of

Re: [VOTE - 2] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

+1 (non-binding) from me I've been a part of the development of this feature throughout its existence, and have reviewed most of the code committed to the branch. The implementation has been conducted carefully and thoughtfully, and tested at a variety of sites. This feature has no potential

[jira] [Resolved] (HDFS-14155) Update "Consistent Read from Observer" User Guide with Edit Tailing Frequency

[ https://issues.apache.org/jira/browse/HDFS-14155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-14155. Resolution: Duplicate > Update "Consistent Read from Observer" User Guide with

[jira] [Created] (HDFS-14155) Update "Consistent Read from Observer" User Guide with Edit Tailing Frequency

Erik Krogen created HDFS-14155: -- Summary: Update "Consistent Read from Observer" User Guide with Edit Tailing Frequency Key: HDFS-14155 URL: https://issues.apache.org/jira/browse/HDFS-14155

[jira] [Resolved] (HDFS-13873) ObserverNode should reject read requests when it is too far behind.

[ https://issues.apache.org/jira/browse/HDFS-13873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-13873. Resolution: Fixed > ObserverNode should reject read requests when it is too far beh

[jira] [Reopened] (HDFS-14048) DFSOutputStream close() throws exception on subsequent call after DataNode restart

[ https://issues.apache.org/jira/browse/HDFS-14048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reopened HDFS-14048: Re-opening for branch-2 commit. Sorry for the trouble [~elgoiri], I have just attached the branch-2

[jira] [Created] (HDFS-14048) DFSOutputStream close() throws exception on subsequent call after DataNode restart

Erik Krogen created HDFS-14048: -- Summary: DFSOutputStream close() throws exception on subsequent call after DataNode restart Key: HDFS-14048 URL: https://issues.apache.org/jira/browse/HDFS-14048 Project

Re: [DISCUSS] Hadoop RPC encryption performance improvements

RPCCallBenchmark. Clearly the performance of the approach is crucial in making the decision and we should ensure that any comparisons made are apples-to-apples with the same test setup. Thanks, Erik Krogen LinkedIn From: Wei-Chiu Chuang Sent: Wednesday, October 31, 2018

[jira] [Created] (HDFS-14034) Support getQuotaUsage API in WebHDFS

Erik Krogen created HDFS-14034: -- Summary: Support getQuotaUsage API in WebHDFS Key: HDFS-14034 URL: https://issues.apache.org/jira/browse/HDFS-14034 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-13977) NameNode can kill itself if it tries to send too many txns to a QJM simultaneously

Erik Krogen created HDFS-13977: -- Summary: NameNode can kill itself if it tries to send too many txns to a QJM simultaneously Key: HDFS-13977 URL: https://issues.apache.org/jira/browse/HDFS-13977 Project

[jira] [Resolved] (HDFS-13930) Fix crlf line endings in HDFS-12943 branch

[ https://issues.apache.org/jira/browse/HDFS-13930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-13930. Resolution: Fixed > Fix crlf line endings in HDFS-12943 bra

[jira] [Created] (HDFS-13930) Fix crlf line endings in HDFS-12943 branch

Erik Krogen created HDFS-13930: -- Summary: Fix crlf line endings in HDFS-12943 branch Key: HDFS-13930 URL: https://issues.apache.org/jira/browse/HDFS-13930 Project: Hadoop HDFS Issue Type: Sub

[jira] [Created] (HDFS-13904) ContentSummary does not always respect processing limit, resulting in long lock acquisitions

Erik Krogen created HDFS-13904: -- Summary: ContentSummary does not always respect processing limit, resulting in long lock acquisitions Key: HDFS-13904 URL: https://issues.apache.org/jira/browse/HDFS-13904

[jira] [Resolved] (HDFS-13872) Only some protocol methods should perform msync wait

[ https://issues.apache.org/jira/browse/HDFS-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-13872. Resolution: Duplicate Closing in favor of HDFS-13880 > Only some protocol methods should perf

[jira] [Created] (HDFS-13872) Only ClientProtocol should perform msync wait

Erik Krogen created HDFS-13872: -- Summary: Only ClientProtocol should perform msync wait Key: HDFS-13872 URL: https://issues.apache.org/jira/browse/HDFS-13872 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-12421) Balancer to emit standard metrics

[ https://issues.apache.org/jira/browse/HDFS-12421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-12421. Resolution: Duplicate Just noticed this is a dup of HDFS-10648   > Balancer to emit stand

[jira] [Created] (HDFS-13791) Limit logging frequency of edit tail related statements

Erik Krogen created HDFS-13791: -- Summary: Limit logging frequency of edit tail related statements Key: HDFS-13791 URL: https://issues.apache.org/jira/browse/HDFS-13791 Project: Hadoop HDFS

[jira] [Created] (HDFS-13789) Reduce logging frequency of QuorumJournalManager#selectInputStreams

Erik Krogen created HDFS-13789: -- Summary: Reduce logging frequency of QuorumJournalManager#selectInputStreams Key: HDFS-13789 URL: https://issues.apache.org/jira/browse/HDFS-13789 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-13150) [Edit Tail Fast Path] Allow SbNN to tail in-progress edits from JN via RPC

[ https://issues.apache.org/jira/browse/HDFS-13150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-13150. Resolution: Fixed Fix Version/s: HDFS-12943 Closing this as all sub-issues (HDFS-13607

[jira] [Created] (HDFS-13689) NameNodeRpcServer getEditsFromTxid assumes it is run on active NameNode

Erik Krogen created HDFS-13689: -- Summary: NameNodeRpcServer getEditsFromTxid assumes it is run on active NameNode Key: HDFS-13689 URL: https://issues.apache.org/jira/browse/HDFS-13689 Project: Hadoop

[jira] [Created] (HDFS-13610) [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

Erik Krogen created HDFS-13610: -- Summary: [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync Key: HDFS-13610 URL: https://issues.apache.org/jira/browse/HDFS-13610

[jira] [Created] (HDFS-13609) [Edit Tail Fast Path Pt 3] NameNode-side changes to support tailing edits via RPC

Erik Krogen created HDFS-13609: -- Summary: [Edit Tail Fast Path Pt 3] NameNode-side changes to support tailing edits via RPC Key: HDFS-13609 URL: https://issues.apache.org/jira/browse/HDFS-13609 Project

[jira] [Created] (HDFS-13608) [Edit Tail Fast Path Pt 2] Add ability for JournalNode to serve edits via RPC

Erik Krogen created HDFS-13608: -- Summary: [Edit Tail Fast Path Pt 2] Add ability for JournalNode to serve edits via RPC Key: HDFS-13608 URL: https://issues.apache.org/jira/browse/HDFS-13608 Project

[jira] [Resolved] (HDFS-13125) Improve efficiency of JN -> Standby Pipeline Under Frequent Edit Tailing

[ https://issues.apache.org/jira/browse/HDFS-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-13125. Resolution: Duplicate This was subsumed by HDFS-13150 > Improve efficiency of JN -> S

[jira] [Created] (HDFS-13602) Optimize checkOperation(WRITE) check in FSNamesystem getBlockLocations

Erik Krogen created HDFS-13602: -- Summary: Optimize checkOperation(WRITE) check in FSNamesystem getBlockLocations Key: HDFS-13602 URL: https://issues.apache.org/jira/browse/HDFS-13602 Project: Hadoop

[jira] [Resolved] (HDFS-13595) Edit tailing period configuration should accept time units

[ https://issues.apache.org/jira/browse/HDFS-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-13595. Resolution: Invalid This is already done. Looked at the wrong branch, my mistake. > Edit tail

[jira] [Created] (HDFS-13595) Edit tailing period configuration should accept time units

Erik Krogen created HDFS-13595: -- Summary: Edit tailing period configuration should accept time units Key: HDFS-13595 URL: https://issues.apache.org/jira/browse/HDFS-13595 Project: Hadoop HDFS

[jira] [Created] (HDFS-13523) Support observer nodes in MiniDFSCluster

Erik Krogen created HDFS-13523: -- Summary: Support observer nodes in MiniDFSCluster Key: HDFS-13523 URL: https://issues.apache.org/jira/browse/HDFS-13523 Project: Hadoop HDFS Issue Type: Sub

[jira] [Created] (HDFS-13522) Support observer node from Router-Based Federation

Erik Krogen created HDFS-13522: -- Summary: Support observer node from Router-Based Federation Key: HDFS-13522 URL: https://issues.apache.org/jira/browse/HDFS-13522 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-13493) Reduce the HttpServer2 thread count on DataNodes

Erik Krogen created HDFS-13493: -- Summary: Reduce the HttpServer2 thread count on DataNodes Key: HDFS-13493 URL: https://issues.apache.org/jira/browse/HDFS-13493 Project: Hadoop HDFS Issue Type

Re: [VOTE] Release Apache Hadoop 2.7.6 (RC0)

Thanks putting this together, Konstantin. +1 (non-binding) * Verified signatures, MD5, and SHA-* checksums for bin and src tarball * Started 2-node HDFS and 1-node YARN clusters * Tested basic file operations, copy(To/From)Local, mkdir, ls, etc. via hdfs and webhdfs protocol * Verified Web UIs

[jira] [Created] (HDFS-13400) WebHDFS append returned stream has incorrectly set position

Erik Krogen created HDFS-13400: -- Summary: WebHDFS append returned stream has incorrectly set position Key: HDFS-13400 URL: https://issues.apache.org/jira/browse/HDFS-13400 Project: Hadoop HDFS

[jira] [Created] (HDFS-13272) DataNodeHttpServer hard-codes HttpServer2 threads at 10

Erik Krogen created HDFS-13272: -- Summary: DataNodeHttpServer hard-codes HttpServer2 threads at 10 Key: HDFS-13272 URL: https://issues.apache.org/jira/browse/HDFS-13272 Project: Hadoop HDFS

[jira] [Created] (HDFS-13265) MiniDFSCluster should set reasonable defaults to reduce resource consumption

Erik Krogen created HDFS-13265: -- Summary: MiniDFSCluster should set reasonable defaults to reduce resource consumption Key: HDFS-13265 URL: https://issues.apache.org/jira/browse/HDFS-13265 Project

[jira] [Created] (HDFS-13264) CacheReplicationMonitor should be able to be disabled completely

Erik Krogen created HDFS-13264: -- Summary: CacheReplicationMonitor should be able to be disabled completely Key: HDFS-13264 URL: https://issues.apache.org/jira/browse/HDFS-13264 Project: Hadoop HDFS

[jira] [Created] (HDFS-13263) DiskBalancer should not start a thread if it is disabled

Erik Krogen created HDFS-13263: -- Summary: DiskBalancer should not start a thread if it is disabled Key: HDFS-13263 URL: https://issues.apache.org/jira/browse/HDFS-13263 Project: Hadoop HDFS

[jira] [Created] (HDFS-13262) Services should not start threads unnecessarily

Erik Krogen created HDFS-13262: -- Summary: Services should not start threads unnecessarily Key: HDFS-13262 URL: https://issues.apache.org/jira/browse/HDFS-13262 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-13150) Create fast path for SbNN tailing edits from JNs

Erik Krogen created HDFS-13150: -- Summary: Create fast path for SbNN tailing edits from JNs Key: HDFS-13150 URL: https://issues.apache.org/jira/browse/HDFS-13150 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-13122) Tailing edits should not update quota counts on ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-13122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-13122. Resolution: Duplicate > Tailing edits should not update quota counts on ObserverN

[jira] [Created] (HDFS-13126) Re-enable HTTP Request Logging for WebHDFS

Erik Krogen created HDFS-13126: -- Summary: Re-enable HTTP Request Logging for WebHDFS Key: HDFS-13126 URL: https://issues.apache.org/jira/browse/HDFS-13126 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-13125) Improve efficiency of JN -> Standby Pipeline Under Frequent Edit Tailing

Erik Krogen created HDFS-13125: -- Summary: Improve efficiency of JN -> Standby Pipeline Under Frequent Edit Tailing Key: HDFS-13125 URL: https://issues.apache.org/jira/browse/HDFS-13125 Project: Had

Announcing Dynamometer Open Sourcing for HDFS Scale Testing

contributions and feedback. Thanks, Erik Krogen HDFS @ LinkedIn [1]: https://github.com/linkedin/dynamometer [2]: https://engineering.linkedin.com/blog/2018/02/dynamometer--scale-testing-hdfs-on-minimal-hardware-with-maximum

[jira] [Created] (HDFS-13122) FSImage should not update quota counts on ObserverNode

Erik Krogen created HDFS-13122: -- Summary: FSImage should not update quota counts on ObserverNode Key: HDFS-13122 URL: https://issues.apache.org/jira/browse/HDFS-13122 Project: Hadoop HDFS Issue

Re: [VOTE] Release Apache Hadoop 2.7.5 (RC1)

+1 (non-binding) • Verified that the missed JIRAs now show up in the release notes • Clicked around the documentation included in the bin tarball • Verified hashes and signatures for the bin and src tarball • Built from source on RHEL 6.6 • Ran small HDFS cluster, executed basic commands, poked

Re: VOTE] Release Apache Hadoop 2.7.5 (RC0)

+1 (non-binding) - Verified signatures, MD5, RMD160, SHA* for bin and src tarballs - Built from source on macOS 10.12.6 and RHEL 6.6 - Ran local HDFS cluster, ran basic commands, verified read and write capability. - Ran 3000 node cluster via Dynamometer and do not see significant performance

[jira] [Created] (HDFS-12828) OIV ReverseXML Processor Fails With Escaped Characters

Erik Krogen created HDFS-12828: -- Summary: OIV ReverseXML Processor Fails With Escaped Characters Key: HDFS-12828 URL: https://issues.apache.org/jira/browse/HDFS-12828 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-12823) Backport HDFS-9259 "Make SO_SNDBUF size configurable at DFSClient" to branch-2.7

Erik Krogen created HDFS-12823: -- Summary: Backport HDFS-9259 "Make SO_SNDBUF size configurable at DFSClient" to branch-2.7 Key: HDFS-12823 URL: https://issues.apache.org/jira/browse/HDFS-12823

[jira] [Created] (HDFS-12818) Support multiple storages in DataNodeCluster / SimulatedFSDataset

Erik Krogen created HDFS-12818: -- Summary: Support multiple storages in DataNodeCluster / SimulatedFSDataset Key: HDFS-12818 URL: https://issues.apache.org/jira/browse/HDFS-12818 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-12803) We should not lock FsNamesystem even we operate a sub directory, we should refinement the lock

[ https://issues.apache.org/jira/browse/HDFS-12803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-12803. Resolution: Duplicate This is a longstanding request tracked in HDFS-5453 > We should not l

[jira] [Created] (HDFS-12746) DataNode Audit Logger

Erik Krogen created HDFS-12746: -- Summary: DataNode Audit Logger Key: HDFS-12746 URL: https://issues.apache.org/jira/browse/HDFS-12746 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Resolved] (HDFS-11707) TestDirectoryScanner#testThrottling fails on OSX

[ https://issues.apache.org/jira/browse/HDFS-11707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-11707. Resolution: Duplicate > TestDirectoryScanner#testThrottling fails on

[jira] [Created] (HDFS-12533) NNThroughputBenchmark threads get stuck on UGI.getCurrentUser()

Erik Krogen created HDFS-12533: -- Summary: NNThroughputBenchmark threads get stuck on UGI.getCurrentUser() Key: HDFS-12533 URL: https://issues.apache.org/jira/browse/HDFS-12533 Project: Hadoop HDFS

  1   2   >