[jira] [Commented] (HDFS-13880) Add mechanism to allow certain RPC calls to bypass sync

[ https://issues.apache.org/jira/browse/HDFS-13880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16609387#comment-16609387 ] Erik Krogen commented on HDFS-13880: [~vagarychen] - {quote} The original thought wa

[jira] [Comment Edited] (HDFS-13880) Add mechanism to allow certain RPC calls to bypass sync

[ https://issues.apache.org/jira/browse/HDFS-13880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16609387#comment-16609387 ] Erik Krogen edited comment on HDFS-13880 at 9/10/18 3:27 PM: -

[jira] [Commented] (HDFS-13898) Throw retriable exception for getBlockLocations when ObserverNameNode is in safemode

[ https://issues.apache.org/jira/browse/HDFS-13898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611150#comment-16611150 ] Erik Krogen commented on HDFS-13898: Nice find [~csun]! The change to production code

[jira] [Comment Edited] (HDFS-13898) Throw retriable exception for getBlockLocations when ObserverNameNode is in safemode

[ https://issues.apache.org/jira/browse/HDFS-13898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611150#comment-16611150 ] Erik Krogen edited comment on HDFS-13898 at 9/11/18 7:47 PM: -

[jira] [Commented] (HDFS-13898) Throw retriable exception for getBlockLocations when ObserverNameNode is in safemode

[ https://issues.apache.org/jira/browse/HDFS-13898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16615283#comment-16615283 ] Erik Krogen commented on HDFS-13898: I explored for a while how to do this without a

[jira] [Commented] (HDFS-13749) Use getServiceStatus to discover observer namenodes

[ https://issues.apache.org/jira/browse/HDFS-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16615391#comment-16615391 ] Erik Krogen commented on HDFS-13749: Nice, v003 looks good to me [~csun]. A few minor

[jira] [Commented] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16746509#comment-16746509 ] Erik Krogen commented on HDFS-14211: {quote} Does that mean every read operation will

[jira] [Commented] (HDFS-14214) IMax and IMin is not work in MutableRatesWithAggregation

[ https://issues.apache.org/jira/browse/HDFS-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16746614#comment-16746614 ] Erik Krogen commented on HDFS-14214: Hi [~xuzq_zander], thanks for reporting this iss

[jira] [Commented] (HDFS-14238) A log in NNThroughputBenchmark should change log level to "INFO" instead of "ERROR"

[ https://issues.apache.org/jira/browse/HDFS-14238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16755307#comment-16755307 ] Erik Krogen commented on HDFS-14238: I could be wrong, but I always interpreted this

[jira] [Commented] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757704#comment-16757704 ] Erik Krogen commented on HDFS-14250: I think #2 is not necessary -- this will be hand

[jira] [Commented] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757722#comment-16757722 ] Erik Krogen commented on HDFS-14250: I don't think we want to make it write anything

[jira] [Commented] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758643#comment-16758643 ] Erik Krogen commented on HDFS-14250: v000 patch LGTM, only minor nit is that I would

[jira] [Commented] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758715#comment-16758715 ] Erik Krogen commented on HDFS-14250: Great ! +1 from me pending Jenkins, will commit

[jira] [Commented] (HDFS-14084) Need for more stats in DFSClient

[ https://issues.apache.org/jira/browse/HDFS-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758718#comment-16758718 ] Erik Krogen commented on HDFS-14084: Hey folks, I was testing out this patch and noti

[jira] [Commented] (HDFS-14084) Need for more stats in DFSClient

[ https://issues.apache.org/jira/browse/HDFS-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758738#comment-16758738 ] Erik Krogen commented on HDFS-14084: Also, if the metrics system _is_ initialized pro

[jira] [Commented] (HDFS-14245) Class cast error in GetGroups with ObserverReadProxyProvider

[ https://issues.apache.org/jira/browse/HDFS-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761968#comment-16761968 ] Erik Krogen commented on HDFS-14245: HAProxy will attempt to create an {{ObserverRead

[jira] [Updated] (HDFS-14250) [SBN read] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14250: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.3.0 Status: Resolved

[jira] [Commented] (HDFS-14250) [SBN read] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761980#comment-16761980 ] Erik Krogen commented on HDFS-14250: The exceptions do not look related. TestBalancer

[jira] [Comment Edited] (HDFS-14250) [SBN read] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761980#comment-16761980 ] Erik Krogen edited comment on HDFS-14250 at 2/6/19 5:55 PM: T

[jira] [Commented] (HDFS-14250) [SBN read] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761988#comment-16761988 ] Erik Krogen commented on HDFS-14250: Jenkins failure does not look related to this pa

[jira] [Updated] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14162: --- Attachment: HDFS-14162.001.patch > Balancer should work with ObserverNode > -

[jira] [Commented] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16763971#comment-16763971 ] Erik Krogen commented on HDFS-14162: I had some discussion offline with [~shv] and he

[jira] [Commented] (HDFS-14262) [SBN read] Unclear Log.WARN message in GlobalStateIdContext

[ https://issues.apache.org/jira/browse/HDFS-14262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16765297#comment-16765297 ] Erik Krogen commented on HDFS-14262: Thanks for the ping [~jojochuang]! I don't think

[jira] [Updated] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14162: --- Attachment: HDFS-14162.002.patch > Balancer should work with ObserverNode > -

[jira] [Commented] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16766553#comment-16766553 ] Erik Krogen commented on HDFS-14162: Thanks [~shv]. I just attached v002 proxy with t

[jira] [Updated] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14162: --- Attachment: ReflectionBenchmark.java > Balancer should work with ObserverNode > -

[jira] [Commented] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767374#comment-16767374 ] Erik Krogen commented on HDFS-14162: I ran a JMH benchmark to see about performance d

[jira] [Comment Edited] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767374#comment-16767374 ] Erik Krogen edited comment on HDFS-14162 at 2/13/19 4:38 PM: -

[jira] [Updated] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14162: --- Attachment: HDFS-14162.003.patch > Balancer should work with ObserverNode > -

[jira] [Commented] (HDFS-14262) [SBN read] Unclear Log.WARN message in GlobalStateIdContext

[ https://issues.apache.org/jira/browse/HDFS-14262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767393#comment-16767393 ] Erik Krogen commented on HDFS-14262: [~brahmareddy] It should never happen. This log

[jira] [Comment Edited] (HDFS-14262) [SBN read] Unclear Log.WARN message in GlobalStateIdContext

[ https://issues.apache.org/jira/browse/HDFS-14262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767393#comment-16767393 ] Erik Krogen edited comment on HDFS-14262 at 2/13/19 5:21 PM: -

[jira] [Assigned] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reassigned HDFS-14272: -- Assignee: Erik Krogen > [SBN read] HDFS command line tools does not guarantee consistency > --

[jira] [Commented] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767418#comment-16767418 ] Erik Krogen commented on HDFS-14272: I don't see us handling this logic anywhere. I t

[jira] [Commented] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767413#comment-16767413 ] Erik Krogen commented on HDFS-14272: Hey [~jojochuang], thanks for reporting. This sh

[jira] [Commented] (HDFS-14271) [SBN read] StandbyException is logged if Observer is the first NameNode

[ https://issues.apache.org/jira/browse/HDFS-14271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767454#comment-16767454 ] Erik Krogen commented on HDFS-14271: Hey [~jojochuang], thanks for reporting. What I

[jira] [Comment Edited] (HDFS-14271) [SBN read] StandbyException is logged if Observer is the first NameNode

[ https://issues.apache.org/jira/browse/HDFS-14271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767454#comment-16767454 ] Erik Krogen edited comment on HDFS-14271 at 2/13/19 6:45 PM: -

[jira] [Updated] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14272: --- Status: Patch Available (was: Open) > [SBN read] HDFS command line tools does not guarantee consiste

[jira] [Updated] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14272: --- Attachment: HDFS-14272.000.patch > [SBN read] HDFS command line tools does not guarantee consistency

[jira] [Commented] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767613#comment-16767613 ] Erik Krogen commented on HDFS-14272: This problem is harder than I thought. Initiall

[jira] [Updated] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14162: --- Attachment: HDFS-14162.004.patch > Balancer should work with ObserverNode > -

[jira] [Commented] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767660#comment-16767660 ] Erik Krogen commented on HDFS-14162: Attached a v004 patch addressing the checkstyle

[jira] [Updated] (HDFS-14262) [SBN read] Unclear Log.WARN message in GlobalStateIdContext

[ https://issues.apache.org/jira/browse/HDFS-14262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14262: --- Fix Version/s: 3.3.0 > [SBN read] Unclear Log.WARN message in GlobalStateIdContext >

[jira] [Updated] (HDFS-14262) [SBN read] Unclear Log.WARN message in GlobalStateIdContext

[ https://issues.apache.org/jira/browse/HDFS-14262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14262: --- Resolution: Fixed Status: Resolved (was: Patch Available) > [SBN read] Unclear Log.WARN mess

[jira] [Commented] (HDFS-14262) [SBN read] Unclear Log.WARN message in GlobalStateIdContext

[ https://issues.apache.org/jira/browse/HDFS-14262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768470#comment-16768470 ] Erik Krogen commented on HDFS-14262: +1 from me. Just committed to trunk. Thanks [~sh

[jira] [Created] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

Erik Krogen created HDFS-14279: -- Summary: [SBN Read] Race condition in ObserverReadProxyProvider Key: HDFS-14279 URL: https://issues.apache.org/jira/browse/HDFS-14279 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14279: --- Attachment: HDFS-14279.000.patch > [SBN Read] Race condition in ObserverReadProxyProvider > -

[jira] [Updated] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14279: --- Status: Patch Available (was: Open) Attaching v000 patch which addresses this problem by having {{ch

[jira] [Commented] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768530#comment-16768530 ] Erik Krogen commented on HDFS-14272: Attached v001 patch which adds some optimization

[jira] [Updated] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14272: --- Attachment: HDFS-14272.001.patch > [SBN read] HDFS command line tools does not guarantee consistency

[jira] [Commented] (HDFS-14276) [SBN read] Reduce tailing overhead

[ https://issues.apache.org/jira/browse/HDFS-14276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768627#comment-16768627 ] Erik Krogen commented on HDFS-14276: Hey [~jojochuang], great find, thanks for report

[jira] [Comment Edited] (HDFS-14277) [SBN read] Observer benchmark results

[ https://issues.apache.org/jira/browse/HDFS-14277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768629#comment-16768629 ] Erik Krogen edited comment on HDFS-14277 at 2/14/19 7:18 PM: -

[jira] [Commented] (HDFS-14277) [SBN read] Observer benchmark results

[ https://issues.apache.org/jira/browse/HDFS-14277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768629#comment-16768629 ] Erik Krogen commented on HDFS-14277: Thanks for reporting this [~jojochuang]! Do you

[jira] [Commented] (HDFS-14277) [SBN read] Observer benchmark results

[ https://issues.apache.org/jira/browse/HDFS-14277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768666#comment-16768666 ] Erik Krogen commented on HDFS-14277: Thanks Wei-Chiu. I don't see any calls to {{isCo

[jira] [Updated] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14162: --- Resolution: Fixed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available) > B

[jira] [Commented] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768668#comment-16768668 ] Erik Krogen commented on HDFS-14162: Just committed to trunk based off of Konstantin'

[jira] [Commented] (HDFS-14277) [SBN read] Observer benchmark results

[ https://issues.apache.org/jira/browse/HDFS-14277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768706#comment-16768706 ] Erik Krogen commented on HDFS-14277: Okay, interesting. Seems that it's at least wort

[jira] [Commented] (HDFS-14284) RBF: Log Router identifier when reporting exceptions

[ https://issues.apache.org/jira/browse/HDFS-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16769714#comment-16769714 ] Erik Krogen commented on HDFS-14284: Interesting problem. Can you maybe post a sample

[jira] [Commented] (HDFS-14284) RBF: Log Router identifier when reporting exceptions

[ https://issues.apache.org/jira/browse/HDFS-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16769816#comment-16769816 ] Erik Krogen commented on HDFS-14284: Makes sense to me. IIRC that information may not

[jira] [Commented] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16772079#comment-16772079 ] Erik Krogen commented on HDFS-14279: The intent is to avoid routing through a {{synch

[jira] [Commented] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

[ https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16772109#comment-16772109 ] Erik Krogen commented on HDFS-14216: Hi [~xiaoheipangzi], thanks for fixing this impo

[jira] [Commented] (HDFS-14297) Add cache for getContentSummary() result

[ https://issues.apache.org/jira/browse/HDFS-14297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16772449#comment-16772449 ] Erik Krogen commented on HDFS-14297: Hi [~Tao Jie], do the directories you are monito

[jira] [Commented] (HDFS-14292) Introduce Java ExecutorService to DataXceiverServer

[ https://issues.apache.org/jira/browse/HDFS-14292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16772457#comment-16772457 ] Erik Krogen commented on HDFS-14292: Hi [~belugabehr], the reasoning makes sense to m

[jira] [Commented] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

[ https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773146#comment-16773146 ] Erik Krogen commented on HDFS-14216: Sure, I see the logic for using DEBUG level. I'm

[jira] [Commented] (HDFS-14297) Add cache for getContentSummary() result

[ https://issues.apache.org/jira/browse/HDFS-14297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773160#comment-16773160 ] Erik Krogen commented on HDFS-14297: Sure, it seems reasonable to me. I see the value

[jira] [Commented] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773262#comment-16773262 ] Erik Krogen commented on HDFS-14272: [~shv], [~vagarychen], [~csun], appreciate if an

[jira] [Updated] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14272: --- Summary: [SBN read] ObserverReadProxyProvider should sync with active txnID on startup (was: [SBN re

[jira] [Comment Edited] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773262#comment-16773262 ] Erik Krogen edited comment on HDFS-14272 at 2/20/19 6:37 PM: -

[jira] [Updated] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14279: --- Attachment: HDFS-14279.001.patch > [SBN Read] Race condition in ObserverReadProxyProvider > -

[jira] [Commented] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773276#comment-16773276 ] Erik Krogen commented on HDFS-14279: I ran some benchmarks offline and found that the

[jira] [Commented] (HDFS-14130) Make ZKFC ObserverNode aware

[ https://issues.apache.org/jira/browse/HDFS-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773366#comment-16773366 ] Erik Krogen commented on HDFS-14130: In the new test, it seems weird to be mixing the

[jira] [Commented] (HDFS-14292) Introduce Java ExecutorService to DataXceiverServer

[ https://issues.apache.org/jira/browse/HDFS-14292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774216#comment-16774216 ] Erik Krogen commented on HDFS-14292: [~kihwal] or [~daryn], I'm curious if either of

[jira] [Commented] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774265#comment-16774265 ] Erik Krogen commented on HDFS-14272: {quote} msync() on active NN if it is a new clie

[jira] [Comment Edited] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774265#comment-16774265 ] Erik Krogen edited comment on HDFS-14272 at 2/21/19 4:28 PM: -

[jira] [Commented] (HDFS-14130) Make ZKFC ObserverNode aware

[ https://issues.apache.org/jira/browse/HDFS-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774286#comment-16774286 ] Erik Krogen commented on HDFS-14130: Agreed that {{System.in}} is not the right appro

[jira] [Commented] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774602#comment-16774602 ] Erik Krogen commented on HDFS-14279: [~shv], does the new patch look good to you? >

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775329#comment-16775329 ] Erik Krogen commented on HDFS-14305: Thanks for reporting this [~csun]. Given that th

[jira] [Updated] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14279: --- Resolution: Fixed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available) > [

[jira] [Commented] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775639#comment-16775639 ] Erik Krogen commented on HDFS-14279: Thanks for the review, [~shv]. I just committed

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777046#comment-16777046 ] Erik Krogen commented on HDFS-14305: The idea seems sound to me. The helper method fo

[jira] [Updated] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14272: --- Attachment: HDFS-14272.002.patch > [SBN read] ObserverReadProxyProvider should sync with active txnID

[jira] [Commented] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777406#comment-16777406 ] Erik Krogen commented on HDFS-14272: [~shv]: {code} Since proxies to individual NNs h

[jira] [Updated] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14272: --- Attachment: (was: HDFS-14272.002.patch) > [SBN read] ObserverReadProxyProvider should sync with a

[jira] [Updated] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14272: --- Attachment: HDFS-14272.002.patch > [SBN read] ObserverReadProxyProvider should sync with active txnID

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778130#comment-16778130 ] Erik Krogen commented on HDFS-14317: Hey [~ekanth], thanks for reporting this. It see

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778137#comment-16778137 ] Erik Krogen commented on HDFS-14305: It looks fine to me. [~vagarychen] has done some

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778226#comment-16778226 ] Erik Krogen commented on HDFS-14317: {quote} I presume the reason why it also removes

[jira] [Updated] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14305: --- Component/s: namenode > Serial number in BlockTokenSecretManager could overlap between different > n

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16779521#comment-16779521 ] Erik Krogen commented on HDFS-14305: v006 patch LGTM. I agree that test failures are

[jira] [Updated] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14305: --- Release Note: NameNodes rely on independent block token key ranges to communicate block token identit

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16779663#comment-16779663 ] Erik Krogen commented on HDFS-14317: Gotcha. Thank you for the detailed explanation -

[jira] [Commented] (HDFS-14084) Need for more stats in DFSClient

[ https://issues.apache.org/jira/browse/HDFS-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16779791#comment-16779791 ] Erik Krogen commented on HDFS-14084: Hi [~pranay_singh], do you plan to complete this

[jira] [Updated] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14305: --- Release Note: NameNodes rely on independent block token key ranges to communicate block token identit

[jira] [Updated] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14305: --- Release Note: NameNodes rely on independent block token key ranges to communicate block token identit

[jira] [Updated] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14305: --- Resolution: Fixed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available) > S

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780742#comment-16780742 ] Erik Krogen commented on HDFS-14305: Thanks [~hexiaoqiao], I agree. I updated the rel

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780746#comment-16780746 ] Erik Krogen commented on HDFS-14305: [~arpitagarwal], thanks for asking, it's a good

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780780#comment-16780780 ] Erik Krogen commented on HDFS-14317: The production code all LGTM. I have a few comme

[jira] [Commented] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780994#comment-16780994 ] Erik Krogen commented on HDFS-12345: Hm. I wonder if it even makes sense for MiniDFSC

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16781094#comment-16781094 ] Erik Krogen commented on HDFS-14317: {quote} Unfortunately, dfs.ha.tail-edits.period

[jira] [Commented] (HDFS-14084) Need for more stats in DFSClient

[ https://issues.apache.org/jira/browse/HDFS-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16781768#comment-16781768 ] Erik Krogen commented on HDFS-14084: Sounds good, thanks [~jojochuang] :) > Need for

<    2   3   4   5   6   7   8   9   10   11   >