[jira] [Commented] (HDFS-13749) Use getServiceStatus to discover observer namenodes

2018-09-05 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605127#comment-16605127 ] Erik Krogen commented on HDFS-13749: Right, sorry for the misdirection, the corresponding class in

[jira] [Comment Edited] (HDFS-13791) Limit logging frequency of edit tail related statements

2018-09-04 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603485#comment-16603485 ] Erik Krogen edited comment on HDFS-13791 at 9/4/18 7:32 PM: Attached v001

[jira] [Commented] (HDFS-13791) Limit logging frequency of edit tail related statements

2018-09-04 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603485#comment-16603485 ] Erik Krogen commented on HDFS-13791: Attached v001 patch with the above discussed changes. I decided

[jira] [Comment Edited] (HDFS-13791) Limit logging frequency of edit tail related statements

2018-09-04 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603485#comment-16603485 ] Erik Krogen edited comment on HDFS-13791 at 9/4/18 7:32 PM: Attached v001

[jira] [Updated] (HDFS-13791) Limit logging frequency of edit tail related statements

2018-09-04 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13791: --- Attachment: HDFS-13791-HDFS-12943.001.patch > Limit logging frequency of edit tail related

[jira] [Commented] (HDFS-13265) MiniDFSCluster should set reasonable defaults to reduce resource consumption

2018-09-04 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603140#comment-16603140 ] Erik Krogen commented on HDFS-13265: Seems that the [branch-2

[jira] [Commented] (HDFS-13791) Limit logging frequency of edit tail related statements

2018-08-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599311#comment-16599311 ] Erik Krogen commented on HDFS-13791: [~csun] thanks for looking! For alignment across logging

[jira] [Commented] (HDFS-13265) MiniDFSCluster should set reasonable defaults to reduce resource consumption

2018-08-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599150#comment-16599150 ] Erik Krogen commented on HDFS-13265: Attaching branch-2 patch to see if this will actually succeed in

[jira] [Updated] (HDFS-13265) MiniDFSCluster should set reasonable defaults to reduce resource consumption

2018-08-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13265: --- Attachment: HDFS-13265-branch-2.001.patch > MiniDFSCluster should set reasonable defaults to reduce

[jira] [Commented] (HDFS-13780) Postpone NameNode state discovery in ObserverReadProxyProvider until the first real RPC call.

2018-08-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599105#comment-16599105 ] Erik Krogen commented on HDFS-13780: Hey [~vagarychen], this was superseded by HDFS-13779, right? >

[jira] [Commented] (HDFS-13880) Add mechanism to allow certain RPC calls to bypass sync

2018-08-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599074#comment-16599074 ] Erik Krogen commented on HDFS-13880: So if I understand correctly, this patch _only_ skips msync on

[jira] [Commented] (HDFS-13872) Only some protocol methods should perform msync wait

2018-08-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599057#comment-16599057 ] Erik Krogen commented on HDFS-13872: Got it... Will move discussion to HDFS-13880, thanks for the

[jira] [Commented] (HDFS-13749) Use getServiceStatus to discover observer namenodes

2018-08-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598956#comment-16598956 ] Erik Krogen commented on HDFS-13749: FYI [~csun], I just committed HDFS-13779, you should be good to

[jira] [Commented] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598955#comment-16598955 ] Erik Krogen commented on HDFS-13779: Good catches, Konstantin! I should have taken a closer look

[jira] [Updated] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13779: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-12943 Status:

[jira] [Updated] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13779: --- Attachment: HDFS-13779-HDFS-12943.003.patch > Implement performFailover logic for

[jira] [Commented] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-30 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598050#comment-16598050 ] Erik Krogen commented on HDFS-13779: Looks like me and Konstantin hit a race condition :) Comments 2

[jira] [Updated] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-30 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13779: --- Attachment: HDFS-13779-HDFS-12943.002.patch > Implement performFailover logic for

[jira] [Assigned] (HDFS-13872) Only some protocol methods should perform msync wait

2018-08-30 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reassigned HDFS-13872: -- Assignee: Erik Krogen > Only some protocol methods should perform msync wait >

[jira] [Work started] (HDFS-13872) Only some protocol methods should perform msync wait

2018-08-30 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-13872 started by Erik Krogen. -- > Only some protocol methods should perform msync wait >

[jira] [Updated] (HDFS-13872) Only some protocol methods should perform msync wait

2018-08-30 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13872: --- Attachment: HDFS-13872-HDFS-12943.000.patch > Only some protocol methods should perform msync wait >

[jira] [Commented] (HDFS-13872) Only some protocol methods should perform msync wait

2018-08-30 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598047#comment-16598047 ] Erik Krogen commented on HDFS-13872: I like the idea of marking only certain methods. I've attached a

[jira] [Updated] (HDFS-13872) Only some protocol methods should perform msync wait

2018-08-30 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13872: --- Summary: Only some protocol methods should perform msync wait (was: Only ClientProtocol should

[jira] [Comment Edited] (HDFS-13749) Use getServiceStatus to discover observer namenodes

2018-08-30 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597916#comment-16597916 ] Erik Krogen edited comment on HDFS-13749 at 8/30/18 9:51 PM: - I think the

[jira] [Commented] (HDFS-13749) Use getServiceStatus to discover observer namenodes

2018-08-30 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597916#comment-16597916 ] Erik Krogen commented on HDFS-13749: I think the idea is sound. I hate to be unable to re-use the

[jira] [Comment Edited] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-30 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597827#comment-16597827 ] Erik Krogen edited comment on HDFS-13779 at 8/30/18 7:34 PM: - Just attached

[jira] [Commented] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-30 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597827#comment-16597827 ] Erik Krogen commented on HDFS-13779: Just attached v001 patch addressing the review comments as

[jira] [Updated] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-30 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13779: --- Attachment: HDFS-13779-HDFS-12943.001.patch > Implement performFailover logic for

[jira] [Commented] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-28 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595743#comment-16595743 ] Erik Krogen commented on HDFS-13779: Thanks for the review [~vagarychen]! # Sure, it is only used for

[jira] [Comment Edited] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594309#comment-16594309 ] Erik Krogen edited comment on HDFS-13779 at 8/27/18 10:50 PM: -- Attached v000

[jira] [Updated] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13779: --- Status: Patch Available (was: In Progress) Attached v000 patch based on recent changes to

[jira] [Updated] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13779: --- Attachment: HDFS-13779-HDFS-12943.000.patch > Implement performFailover logic for

[jira] [Comment Edited] (HDFS-13749) Implement a new client protocol method to get NameNode state

2018-08-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594291#comment-16594291 ] Erik Krogen edited comment on HDFS-13749 at 8/27/18 10:29 PM: -- Hey [~csun],

[jira] [Commented] (HDFS-13749) Implement a new client protocol method to get NameNode state

2018-08-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594291#comment-16594291 ] Erik Krogen commented on HDFS-13749: Hey [~csun], seems we can close this out given that we committed

[jira] [Commented] (HDFS-13872) Only ClientProtocol should perform msync wait

2018-08-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594288#comment-16594288 ] Erik Krogen commented on HDFS-13872: [~vagarychen], [~shv], thoughts? > Only ClientProtocol should

[jira] [Created] (HDFS-13872) Only ClientProtocol should perform msync wait

2018-08-27 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-13872: -- Summary: Only ClientProtocol should perform msync wait Key: HDFS-13872 URL: https://issues.apache.org/jira/browse/HDFS-13872 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-13848) Refactor NameNode failover proxy providers

2018-08-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593792#comment-16593792 ] Erik Krogen commented on HDFS-13848: {quote} addressKey is a hacky solution introduced in HDFS-13536

[jira] [Commented] (HDFS-13782) ObserverReadProxyProvider should work with IPFailoverProxyProvider

2018-08-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593789#comment-16593789 ] Erik Krogen commented on HDFS-13782: {quote} {{ObserverReadProxyProvider}} is not tied to CFPP,

[jira] [Commented] (HDFS-13759) [HDFS Pagination]Does HDFS Java api Support Pagination?

2018-08-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592042#comment-16592042 ] Erik Krogen commented on HDFS-13759: The {{DFSClient}} is already doing this under the hood (see

[jira] [Updated] (HDFS-13791) Limit logging frequency of edit tail related statements

2018-08-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13791: --- Status: Patch Available (was: In Progress) Attaching a patch which takes the same logic as added in

[jira] [Updated] (HDFS-13791) Limit logging frequency of edit tail related statements

2018-08-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13791: --- Attachment: HDFS-13791-HDFS-12943.000.patch > Limit logging frequency of edit tail related

[jira] [Comment Edited] (HDFS-13782) ObserverReadProxyProvider should work with IPFailoverProxyProvider

2018-08-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591872#comment-16591872 ] Erik Krogen edited comment on HDFS-13782 at 8/24/18 4:31 PM: - Besides

[jira] [Commented] (HDFS-13782) ObserverReadProxyProvider should work with IPFailoverProxyProvider

2018-08-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591872#comment-16591872 ] Erik Krogen commented on HDFS-13782: Besides rebasing on top of HDFS-13848 once it is committed, this

[jira] [Updated] (HDFS-13851) Remove AlignmentContext from AbstractNNFailoverProxyProvider

2018-08-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13851: --- Resolution: Fixed Fix Version/s: HDFS-12943 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-13848) Refactor NameNode failover proxy providers

2018-08-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591571#comment-16591571 ] Erik Krogen commented on HDFS-13848: Cool, thanks for the quick turnaround [~shv]! Overall things

[jira] [Commented] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-23 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590979#comment-16590979 ] Erik Krogen commented on HDFS-13779: I had some offline discussion with [~shv] and we realized that

[jira] [Commented] (HDFS-13848) Refactor NameNode failover proxy providers

2018-08-23 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590972#comment-16590972 ] Erik Krogen commented on HDFS-13848: Hey [~shv], I think this is a great consolidation of redundant

[jira] [Commented] (HDFS-13851) Remove AlignmentContext from AbstractNNFailoverProxyProvider

2018-08-23 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590957#comment-16590957 ] Erik Krogen commented on HDFS-13851: +1 from me as well > Remove AlignmentContext from

[jira] [Work started] (HDFS-13791) Limit logging frequency of edit tail related statements

2018-08-21 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-13791 started by Erik Krogen. -- > Limit logging frequency of edit tail related statements >

[jira] [Commented] (HDFS-13820) Disable CacheReplicationMonitor If No Cached Paths Exist

2018-08-17 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584246#comment-16584246 ] Erik Krogen commented on HDFS-13820: +1, we have seen from the FSNamesystem metrics added in

[jira] [Commented] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-16 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583012#comment-16583012 ] Erik Krogen commented on HDFS-13779: {quote} And {{getProxy}} will trigger {{getAllProxies}} which

[jira] [Comment Edited] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-15 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581593#comment-16581593 ] Erik Krogen edited comment on HDFS-13779 at 8/15/18 8:53 PM: - Thanks for

[jira] [Commented] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-15 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581593#comment-16581593 ] Erik Krogen commented on HDFS-13779: Thanks for looking [~vagarychen]! For #1, the {{currentIndex}}

[jira] [Commented] (HDFS-13783) Balancer: make balancer to be a long service process for easy to monitor it.

2018-08-09 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575360#comment-16575360 ] Erik Krogen commented on HDFS-13783: This would be helpful for HDFS-10648! :) > Balancer: make

[jira] [Resolved] (HDFS-12421) Balancer to emit standard metrics

2018-08-09 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-12421. Resolution: Duplicate Just noticed this is a dup of HDFS-10648   > Balancer to emit standard

[jira] [Updated] (HDFS-10648) Expose Balancer metrics through Metrics2

2018-08-09 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10648: --- Component/s: metrics > Expose Balancer metrics through Metrics2 >

[jira] [Commented] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574081#comment-16574081 ] Erik Krogen commented on HDFS-13779: Attaching a WIP patch which covers this and HDFS-13780, and

[jira] [Updated] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13779: --- Attachment: HDFS-13779-HDFS-12943.WIP00.patch > Implement performFailover logic for

[jira] [Commented] (HDFS-13566) Add configurable additional RPC listener to NameNode

2018-08-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573443#comment-16573443 ] Erik Krogen commented on HDFS-13566: Cool, thanks for clarifying. My initial reaction to (3) was that

[jira] [Commented] (HDFS-13566) Add configurable additional RPC listener to NameNode

2018-08-07 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571903#comment-16571903 ] Erik Krogen commented on HDFS-13566: [~shv] - I think the idea is sound, but I have a question about

[jira] [Commented] (HDFS-12976) Introduce ObserverReadProxyProvider

2018-08-03 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568861#comment-16568861 ] Erik Krogen commented on HDFS-12976: Hey [~csun], one follow-up question about these two lines:

[jira] [Work started] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-03 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-13779 started by Erik Krogen. -- > Implement performFailover logic for ObserverReadProxyProvider. >

[jira] [Commented] (HDFS-13789) Reduce logging frequency of QuorumJournalManager#selectInputStreams

2018-08-03 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568589#comment-16568589 ] Erik Krogen commented on HDFS-13789: Hey [~csun], thanks for taking a look. Yes, I agree we should do

[jira] [Created] (HDFS-13791) Limit logging frequency of edit tail related statements

2018-08-03 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-13791: -- Summary: Limit logging frequency of edit tail related statements Key: HDFS-13791 URL: https://issues.apache.org/jira/browse/HDFS-13791 Project: Hadoop HDFS

[jira] [Updated] (HDFS-13782) ObserverReadProxyProvider should work with IPFailoverProxyProvider

2018-08-03 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13782: --- Summary: ObserverReadProxyProvider should work with IPFailoverProxyProvider (was:

[jira] [Assigned] (HDFS-13779) Implement performFailover logic for ObserverReadProxyProvider.

2018-08-03 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reassigned HDFS-13779: -- Assignee: Erik Krogen > Implement performFailover logic for ObserverReadProxyProvider. >

[jira] [Updated] (HDFS-13789) Reduce logging frequency of QuorumJournalManager#selectInputStreams

2018-08-02 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13789: --- Status: Patch Available (was: In Progress) > Reduce logging frequency of

[jira] [Updated] (HDFS-13789) Reduce logging frequency of QuorumJournalManager#selectInputStreams

2018-08-02 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13789: --- Attachment: HDFS-13789-HDFS-12943.000.patch > Reduce logging frequency of

[jira] [Work started] (HDFS-13789) Reduce logging frequency of QuorumJournalManager#selectInputStreams

2018-08-02 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-13789 started by Erik Krogen. -- > Reduce logging frequency of QuorumJournalManager#selectInputStreams >

[jira] [Updated] (HDFS-13789) Reduce logging frequency of QuorumJournalManager#selectInputStreams

2018-08-02 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13789: --- Issue Type: Sub-task (was: Improvement) Parent: HDFS-12943 > Reduce logging frequency of

[jira] [Created] (HDFS-13789) Reduce logging frequency of QuorumJournalManager#selectInputStreams

2018-08-02 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-13789: -- Summary: Reduce logging frequency of QuorumJournalManager#selectInputStreams Key: HDFS-13789 URL: https://issues.apache.org/jira/browse/HDFS-13789 Project: Hadoop HDFS

[jira] [Commented] (HDFS-13767) Add msync server implementation.

2018-08-02 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567060#comment-16567060 ] Erik Krogen commented on HDFS-13767: Cool, thanks for incorporating the feedback Chen! I agree with

[jira] [Updated] (HDFS-13688) Introduce msync API call

2018-08-01 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13688: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-12943 Status:

[jira] [Commented] (HDFS-13767) Add msync server implementation.

2018-07-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563928#comment-16563928 ] Erik Krogen commented on HDFS-13767: I agree with Konstantin's point that maintaining the ordering is

[jira] [Commented] (HDFS-13688) Introduce msync API call

2018-07-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563873#comment-16563873 ] Erik Krogen commented on HDFS-13688: Cool, v005 patch LGTM. I'll commit EOD today in case [~shv]

[jira] [Commented] (HDFS-13688) Introduce msync API call

2018-07-30 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562631#comment-16562631 ] Erik Krogen commented on HDFS-13688: Hey [~vagarychen], I think separating out into client- and

[jira] [Commented] (HDFS-13523) Support observer nodes in MiniDFSCluster

2018-07-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560332#comment-16560332 ] Erik Krogen commented on HDFS-13523: Thanks for working on this [~zhengxg3]! Glad we finally got the

[jira] [Commented] (HDFS-13553) RBF: Support global quota

2018-07-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554495#comment-16554495 ] Erik Krogen commented on HDFS-13553: Hi [~elgoiri], I see that with HDFS-13583 being closed all

[jira] [Resolved] (HDFS-13150) [Edit Tail Fast Path] Allow SbNN to tail in-progress edits from JN via RPC

2018-07-17 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-13150. Resolution: Fixed Fix Version/s: HDFS-12943 Closing this as all sub-issues (HDFS-13607,

[jira] [Commented] (HDFS-13610) [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

2018-07-17 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546979#comment-16546979 ] Erik Krogen commented on HDFS-13610: I just committed this based on [~shv]'s +1. Confirmed that the

[jira] [Updated] (HDFS-13610) [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

2018-07-17 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13610: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-12943 Status:

[jira] [Commented] (HDFS-13523) Support observer nodes in MiniDFSCluster

2018-07-17 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546859#comment-16546859 ] Erik Krogen commented on HDFS-13523: Thanks for working on this [~zhengxg3]! A few comments: * Can we

[jira] [Commented] (HDFS-13735) Make QJM HTTP URL connection timeout configurable

2018-07-16 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545822#comment-16545822 ] Erik Krogen commented on HDFS-13735: The logic seems good to me. It is a pretty general change so I

[jira] [Commented] (HDFS-13735) Make QJM HTTP URL connection timeout configurable

2018-07-16 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545678#comment-16545678 ] Erik Krogen commented on HDFS-13735: The patch LGTM except that you need to update

[jira] [Commented] (HDFS-13610) [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

2018-07-16 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545532#comment-16545532 ] Erik Krogen commented on HDFS-13610: Sorry for my late reply; I just returned from vacation.

[jira] [Updated] (HDFS-13610) [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

2018-07-16 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13610: --- Attachment: HDFS-13610-HDFS-12943.004.patch > [Edit Tail Fast Path Pt 4] Cleanup: integration test,

[jira] [Commented] (HDFS-13706) ClientGCIContext should be correctly named ClientGSIContext

2018-06-29 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527490#comment-16527490 ] Erik Krogen commented on HDFS-13706: +1 I have been wondering what GCI stood for >

[jira] [Updated] (HDFS-13610) [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

2018-06-29 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13610: --- Description: See HDFS-13150 for full design. This JIRA is targeted at cleanup tasks: * Add in

[jira] [Updated] (HDFS-13610) [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

2018-06-29 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13610: --- Description: See HDFS-13150 for full design. This JIRA is targeted at cleanup tasks: * Add in

[jira] [Commented] (HDFS-13665) Move RPC response serialization into Server.doResponse

2018-06-29 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527256#comment-16527256 ] Erik Krogen commented on HDFS-13665: (1) Sounds good, I gathered the same insight but wanted a second

[jira] [Commented] (HDFS-13665) Move RPC response serialization into Server.doResponse

2018-06-28 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526292#comment-16526292 ] Erik Krogen commented on HDFS-13665: Hey [~zero45], the intent of the change looks good to me. I have

[jira] [Comment Edited] (HDFS-13610) [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

2018-06-28 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526153#comment-16526153 ] Erik Krogen edited comment on HDFS-13610 at 6/28/18 10:05 AM: -- Thanks for

[jira] [Commented] (HDFS-13610) [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

2018-06-28 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526153#comment-16526153 ] Erik Krogen commented on HDFS-13610: Thanks for the review [~linyiqun]! * Done, good idea. * I

[jira] [Updated] (HDFS-13610) [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

2018-06-28 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13610: --- Attachment: HDFS-13610-HDFS-12943.003.patch > [Edit Tail Fast Path Pt 4] Cleanup: integration test,

[jira] [Commented] (HDFS-12976) Introduce ObserverReadProxyProvider

2018-06-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525875#comment-16525875 ] Erik Krogen commented on HDFS-12976: I think to pursue an approach like [~shv]'s, we need to define

[jira] [Comment Edited] (HDFS-13617) Allow wrapping NN QOP into token in encrypted message

2018-06-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524793#comment-16524793 ] Erik Krogen edited comment on HDFS-13617 at 6/27/18 9:16 AM: - Hey

[jira] [Comment Edited] (HDFS-13617) Allow wrapping NN QOP into token in encrypted message

2018-06-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524793#comment-16524793 ] Erik Krogen edited comment on HDFS-13617 at 6/27/18 9:15 AM: - Hey

[jira] [Commented] (HDFS-13617) Allow wrapping NN QOP into token in encrypted message

2018-06-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524793#comment-16524793 ] Erik Krogen commented on HDFS-13617: Hey [~vagarychen], I just tried to apply this on trunk and the

[jira] [Updated] (HDFS-13610) [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

2018-06-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13610: --- Attachment: HDFS-13610-HDFS-12943.002.patch > [Edit Tail Fast Path Pt 4] Cleanup: integration test,

[jira] [Commented] (HDFS-13610) [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

2018-06-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524657#comment-16524657 ] Erik Krogen commented on HDFS-13610: Looking through the test failures, I noticed some testing

<    5   6   7   8   9   10   11   12   13   14   >