[jira] [Updated] (HDFS-14366) Improve HDFS append performance

2019-03-12 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14366: Description: In our HDFS cluster we observed that {{append}} operation can take as much as 10X write lock

[jira] [Updated] (HDFS-14366) Improve HDFS append performance

2019-03-12 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14366: Attachment: append-flamegraph.png > Improve HDFS append performance > --- > >

[jira] [Created] (HDFS-14366) Improve HDFS append performance

2019-03-12 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14366: --- Summary: Improve HDFS append performance Key: HDFS-14366 URL: https://issues.apache.org/jira/browse/HDFS-14366 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Commented] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

2019-03-11 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789811#comment-16789811 ] Chao Sun commented on HDFS-14211: - One potential downside with this approach, IMO, is that {{msync}}

[jira] [Commented] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-03-11 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789807#comment-16789807 ] Chao Sun commented on HDFS-14205: - Turned out I need to generate the patch using {{--binary}} flag. There

[jira] [Commented] (HDFS-14346) Better time precision in getTimeDuration

2019-03-11 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789787#comment-16789787 ] Chao Sun commented on HDFS-14346: - Thanks [~xkrogen] for the reviewing! changed the title and

[jira] [Updated] (HDFS-14346) Better time precision in getTimeDuration

2019-03-11 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14346: Description: Currently, {{Configuration#getTimeDuration}} has the following signature: {code} /** *

[jira] [Updated] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-03-11 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14205: Attachment: HDFS-14205-branch-2.009.patch > Backport HDFS-6440 to branch-2 >

[jira] [Updated] (HDFS-14346) EditLogTailer loses precision for sub-second edit log tailing and rolling interval

2019-03-11 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14346: Status: Patch Available (was: Open) > EditLogTailer loses precision for sub-second edit log tailing and

[jira] [Updated] (HDFS-14346) EditLogTailer loses precision for sub-second edit log tailing and rolling interval

2019-03-11 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14346: Attachment: HDFS-14346.000.patch > EditLogTailer loses precision for sub-second edit log tailing and

[jira] [Updated] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-03-07 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14205: Attachment: HDFS-14205-branch-2.008.patch > Backport HDFS-6440 to branch-2 >

[jira] [Commented] (HDFS-14346) EditLogTailer loses precision for sub-second edit log tailing and rolling interval

2019-03-07 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787231#comment-16787231 ] Chao Sun commented on HDFS-14346: - Thanks [~xkrogen]! You are totally right - we need be careful to

[jira] [Created] (HDFS-14346) EditLogTailer loses precision for sub-second edit log tailing and rolling interval

2019-03-07 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14346: --- Summary: EditLogTailer loses precision for sub-second edit log tailing and rolling interval Key: HDFS-14346 URL: https://issues.apache.org/jira/browse/HDFS-14346 Project:

[jira] [Commented] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-03-04 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784124#comment-16784124 ] Chao Sun commented on HDFS-14205: - Hmm... all the failed tests are passing locally on my laptop..

[jira] [Commented] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-03-04 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783817#comment-16783817 ] Chao Sun commented on HDFS-14205: - Re-attach patch v6 to trigger jenkins. > Backport HDFS-6440 to

[jira] [Updated] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-03-04 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14205: Attachment: HDFS-14205-branch-2.007.patch > Backport HDFS-6440 to branch-2 >

[jira] [Updated] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-03-01 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14205: Attachment: HDFS-14205-branch-2.006.patch > Backport HDFS-6440 to branch-2 >

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-03-01 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782170#comment-16782170 ] Chao Sun commented on HDFS-14317: - Regarding the time unit issue for log rolling and tailing, perhaps we

[jira] [Updated] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-02-28 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14205: Attachment: HDFS-14205-branch-2.005.patch > Backport HDFS-6440 to branch-2 >

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-28 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16780799#comment-16780799 ] Chao Sun commented on HDFS-14305: - [~arpitagarwal] yes like [~xkrogen] said, without this fix, a

[jira] [Commented] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-02-28 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16780798#comment-16780798 ] Chao Sun commented on HDFS-14205: - Oops a typo in v4. Uploaded patch v5. > Backport HDFS-6440 to

[jira] [Updated] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-02-28 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14317: Status: Patch Available (was: Open) > Standby does not trigger edit log rolling when in-progress edit log

[jira] [Updated] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-02-27 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14205: Attachment: HDFS-14205-branch-2.004.patch > Backport HDFS-6440 to branch-2 >

[jira] [Commented] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-02-26 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778876#comment-16778876 ] Chao Sun commented on HDFS-14205: - Thanks [~vagarychen]. Jenkins is working now. I'll investigate the

[jira] [Commented] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-02-26 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778723#comment-16778723 ] Chao Sun commented on HDFS-14205: - Attached patch v3. The main change is that we forgot to use

[jira] [Comment Edited] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-02-26 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778723#comment-16778723 ] Chao Sun edited comment on HDFS-14205 at 2/27/19 12:02 AM: --- Attached patch v3.

[jira] [Updated] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-02-26 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14205: Attachment: HDFS-14205-branch-2.003.patch > Backport HDFS-6440 to branch-2 >

[jira] [Commented] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-02-26 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778475#comment-16778475 ] Chao Sun commented on HDFS-14205: - Sorry for the delay [~vagarychen]. I'm working on an updated patch

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-26 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778415#comment-16778415 ] Chao Sun commented on HDFS-14305: - Patch looks good to me as well. Thanks [~hexiaoqiao]! Some nits: #

[jira] [Assigned] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-25 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HDFS-14305: --- Assignee: He Xiaoqiao (was: Chao Sun) > Serial number in BlockTokenSecretManager could overlap

[jira] [Commented] (HDFS-14130) Make ZKFC ObserverNode aware

2019-02-22 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775709#comment-16775709 ] Chao Sun commented on HDFS-14130: - +1 on patch v11. > Make ZKFC ObserverNode aware >

[jira] [Commented] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

2019-02-22 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775664#comment-16775664 ] Chao Sun commented on HDFS-14272: - Thanks [~xkrogen]. The change on {{msync}} looks good to me. Some very

[jira] [Commented] (HDFS-14130) Make ZKFC ObserverNode aware

2019-02-22 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775470#comment-16775470 ] Chao Sun commented on HDFS-14130: - Thanks [~shv]. I'm slightly more in favor of the System.in approach

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-22 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775439#comment-16775439 ] Chao Sun commented on HDFS-14305: - Thanks [~hexiaoqiao]. One potential issue with the patch 001 is that

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-21 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774839#comment-16774839 ] Chao Sun commented on HDFS-14305: - [~jojochuang] this problem is not related to observer nodes - it could

[jira] [Updated] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-21 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14305: Issue Type: Bug (was: Improvement) > Serial number in BlockTokenSecretManager could overlap between

[jira] [Created] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-21 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14305: --- Summary: Serial number in BlockTokenSecretManager could overlap between different namenodes Key: HDFS-14305 URL: https://issues.apache.org/jira/browse/HDFS-14305 Project:

[jira] [Comment Edited] (HDFS-14130) Make ZKFC ObserverNode aware

2019-02-21 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774354#comment-16774354 ] Chao Sun edited comment on HDFS-14130 at 2/21/19 5:51 PM: -- [~xkrogen]: yes, I

[jira] [Commented] (HDFS-14130) Make ZKFC ObserverNode aware

2019-02-21 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774354#comment-16774354 ] Chao Sun commented on HDFS-14130: - [~xkrogen]: yes, I will also take a look at the this today. > Make

[jira] [Commented] (HDFS-14130) Make ZKFC ObserverNode aware

2019-02-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16771465#comment-16771465 ] Chao Sun commented on HDFS-14130: - Seems the test failure on \{{testManualFailoverWithDFSHAAdmin}} is

[jira] [Commented] (HDFS-14284) RBF: Log Router identifier when reporting exceptions

2019-02-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16771453#comment-16771453 ] Chao Sun commented on HDFS-14284: - Agree this could be useful. Perhaps we can make this optional by

[jira] [Commented] (HDFS-14276) [SBN read] Reduce tailing overhead

2019-02-14 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16768685#comment-16768685 ] Chao Sun commented on HDFS-14276: - Not sure if I understand this correctly. With in-progress edit log

[jira] [Updated] (HDFS-14250) [SBN read] msync should sync with active NameNode to fetch the latest stateID

2019-02-01 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14250: Summary: [SBN read] msync should sync with active NameNode to fetch the latest stateID (was: [Standby

[jira] [Updated] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

2019-02-01 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14250: Attachment: HDFS-14250.001.patch > [Standby Reads] msync should sync with active NameNode to fetch the

[jira] [Commented] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

2019-02-01 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758707#comment-16758707 ] Chao Sun commented on HDFS-14250: - Thanks [~xkrogen]! Attached patch v1 to address the comment. >

[jira] [Updated] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

2019-02-01 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14250: Attachment: HDFS-14250.000.patch > [Standby Reads] msync should sync with active NameNode to fetch the

[jira] [Updated] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

2019-02-01 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14250: Status: Patch Available (was: Open) > [Standby Reads] msync should sync with active NameNode to fetch the

[jira] [Commented] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

2019-01-31 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757742#comment-16757742 ] Chao Sun commented on HDFS-14250: - Yes, I think we can just use {{activeOnly}} for this type of purpose.

[jira] [Commented] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

2019-01-31 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757712#comment-16757712 ] Chao Sun commented on HDFS-14250: - [~xkrogen] you are right. Maybe {{msync}} can simply be a write call

[jira] [Created] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

2019-01-31 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14250: --- Summary: [Standby Reads] msync should sync with active NameNode to fetch the latest stateID Key: HDFS-14250 URL: https://issues.apache.org/jira/browse/HDFS-14250 Project:

[jira] [Commented] (HDFS-14118) Use DNS to resolve Namenodes and Routers

2019-01-29 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755533#comment-16755533 ] Chao Sun commented on HDFS-14118: - I'm not sure if we need any new config in {{core-default.xml}} for

[jira] [Commented] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-01-29 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755233#comment-16755233 ] Chao Sun commented on HDFS-14205: - [~vagarychen] seems the patch no longer applies to branch-2. Could you

[jira] [Commented] (HDFS-14118) Use DNS to resolve Namenodes and Routers

2019-01-28 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754659#comment-16754659 ] Chao Sun commented on HDFS-14118: - Thanks [~fengnanli]. Patch v5 looks better. Some (most minor)

[jira] [Commented] (HDFS-14130) Make ZKFC ObserverNode aware

2019-01-28 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754378#comment-16754378 ] Chao Sun commented on HDFS-14130: - Thanks for the update [~xiangheng]. The test failure seems related.

[jira] [Commented] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-01-28 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754247#comment-16754247 ] Chao Sun commented on HDFS-14205: - Cool. Thanks [~vagarychen]. I can take a look at these. Also we may

[jira] [Assigned] (HDFS-14205) Backport HDFS-6440 to branch-2

2019-01-26 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HDFS-14205: --- Assignee: Chao Sun > Backport HDFS-6440 to branch-2 > -- > >

[jira] [Commented] (HDFS-14118) Use DNS to resolve Namenodes and Routers

2019-01-23 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750322#comment-16750322 ] Chao Sun commented on HDFS-14118: - I think overall it makes sense to have the resolver pluggable so other

[jira] [Commented] (HDFS-14130) Make ZKFC ObserverNode aware

2019-01-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745822#comment-16745822 ] Chao Sun commented on HDFS-14130: - Thanks [~xiangheng]. I think you may need to add the test in

[jira] [Commented] (HDFS-14130) Make ZKFC ObserverNode aware

2019-01-16 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744735#comment-16744735 ] Chao Sun commented on HDFS-14130: - Hi [~xiangheng]: this is not related to HDFS-14067. What I mean is a

[jira] [Commented] (HDFS-14130) Make ZKFC ObserverNode aware

2019-01-16 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744580#comment-16744580 ] Chao Sun commented on HDFS-14130: - [~xiangheng]: yes that's what I mean. > Make ZKFC ObserverNode aware

[jira] [Commented] (HDFS-14130) Make ZKFC ObserverNode aware

2019-01-16 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744544#comment-16744544 ] Chao Sun commented on HDFS-14130: - Hi [~xiangheng]: thanks for working on this, the patch looks good. I

[jira] [Commented] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

2019-01-16 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744336#comment-16744336 ] Chao Sun commented on HDFS-14211: - Thanks for filing this [~xkrogen]! I think this will be a very useful

[jira] [Commented] (HDFS-14168) Fix TestWebHdfsTimeouts

2018-12-31 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16731401#comment-16731401 ] Chao Sun commented on HDFS-14168: - Oh, cool! I'll mark this as a duplicate then. > Fix

[jira] [Resolved] (HDFS-14168) Fix TestWebHdfsTimeouts

2018-12-31 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HDFS-14168. - Resolution: Duplicate > Fix TestWebHdfsTimeouts > --- > > Key:

[jira] [Commented] (HDFS-11043) TestWebHdfsTimeouts fails

2018-12-22 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16727571#comment-16727571 ] Chao Sun commented on HDFS-11043: - Seems this test is still keep failing. Filed HDFS-14168 to track it.

[jira] [Created] (HDFS-14168) Fix TestWebHdfsTimeouts

2018-12-22 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14168: --- Summary: Fix TestWebHdfsTimeouts Key: HDFS-14168 URL: https://issues.apache.org/jira/browse/HDFS-14168 Project: Hadoop HDFS Issue Type: Test Components:

[jira] [Assigned] (HDFS-13522) Support observer node from Router-Based Federation

2018-12-22 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HDFS-13522: --- Assignee: Chao Sun > Support observer node from Router-Based Federation >

[jira] [Assigned] (HDFS-13152) Support observer reads for WebHDFS

2018-12-22 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HDFS-13152: --- Assignee: Chao Sun > Support observer reads for WebHDFS > --- > >

[jira] [Updated] (HDFS-14154) Document dfs.ha.tail-edits.period in user guide.

2018-12-19 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14154: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-12943 Status: Resolved

[jira] [Commented] (HDFS-14154) Document dfs.ha.tail-edits.period in user guide.

2018-12-19 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725303#comment-16725303 ] Chao Sun commented on HDFS-14154: - Oops. My bad. > Document dfs.ha.tail-edits.period in user guide. >

[jira] [Updated] (HDFS-14154) Document dfs.ha.tail-edits.period in user guide.

2018-12-19 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14154: Attachment: (was: HDFS-14154-HDFS-12943.001.patch) > Document dfs.ha.tail-edits.period in user guide.

[jira] [Updated] (HDFS-14154) Document dfs.ha.tail-edits.period in user guide.

2018-12-19 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14154: Attachment: HDFS-14154-HDFS-12943.001.patch > Document dfs.ha.tail-edits.period in user guide. >

[jira] [Commented] (HDFS-14160) ObserverReadInvocationHandler should implement RpcInvocationHandler

2018-12-19 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725283#comment-16725283 ] Chao Sun commented on HDFS-14160: - [~shv]: Hadoop QA is complaining about an extra whitespace. Could you

[jira] [Updated] (HDFS-14154) Document dfs.ha.tail-edits.period in user guide.

2018-12-19 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14154: Attachment: HDFS-14154-HDFS-12943.001.patch > Document dfs.ha.tail-edits.period in user guide. >

[jira] [Commented] (HDFS-14154) Document dfs.ha.tail-edits.period in user guide.

2018-12-19 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725281#comment-16725281 ] Chao Sun commented on HDFS-14154: - Attached patch v1 to address comments. > Document

[jira] [Commented] (HDFS-14154) Document dfs.ha.tail-edits.period in user guide.

2018-12-19 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725213#comment-16725213 ] Chao Sun commented on HDFS-14154: - Thanks [~xkrogen] for the great comments. Will fix the typo and

[jira] [Commented] (HDFS-14154) Document dfs.ha.tail-edits.period in user guide.

2018-12-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724582#comment-16724582 ] Chao Sun commented on HDFS-14154: - Attached patch v0. > Document dfs.ha.tail-edits.period in user guide.

[jira] [Updated] (HDFS-14154) Document dfs.ha.tail-edits.period in user guide.

2018-12-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14154: Attachment: HDFS-14154-HDFS-12943.000.patch > Document dfs.ha.tail-edits.period in user guide. >

[jira] [Updated] (HDFS-14154) Document dfs.ha.tail-edits.period in user guide.

2018-12-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14154: Status: Patch Available (was: Open) > Document dfs.ha.tail-edits.period in user guide. >

[jira] [Commented] (HDFS-14160) ObserverReadInvocationHandler should implement RpcInvocationHandler

2018-12-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724541#comment-16724541 ] Chao Sun commented on HDFS-14160: - +1. LGTM with pending tests. > ObserverReadInvocationHandler should

[jira] [Commented] (HDFS-14116) Fix class cast error in NNThroughputBenchmark with ObserverReadProxyProvider.

2018-12-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723752#comment-16723752 ] Chao Sun commented on HDFS-14116: - OK. I was just concerned that in some edge cases people may be caught

[jira] [Commented] (HDFS-12943) Consistent Reads from Standby Node

2018-12-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723757#comment-16723757 ] Chao Sun commented on HDFS-12943: - [~brahmareddy] [~xkrogen]: unfortunately I can't provide enough data

[jira] [Commented] (HDFS-14116) Fix class cast error in NNThroughputBenchmark with ObserverReadProxyProvider.

2018-12-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723526#comment-16723526 ] Chao Sun commented on HDFS-14116: - Thanks [~shv]. I'm fine with forbidding using ORPP with any protocol

[jira] [Updated] (HDFS-14154) Document dfs.ha.tail-edits.period in user guide.

2018-12-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14154: Summary: Document dfs.ha.tail-edits.period in user guide. (was: Add explanation for

[jira] [Commented] (HDFS-12943) Consistent Reads from Standby Node

2018-12-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723278#comment-16723278 ] Chao Sun commented on HDFS-12943: - I think we should document {{dfs.ha.tail-edits.period}} in the user

[jira] [Updated] (HDFS-14154) Add explanation for dfs.ha.tail-edits.period in user guide.

2018-12-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14154: Issue Type: Sub-task (was: New Feature) Parent: HDFS-12943 > Add explanation for

[jira] [Created] (HDFS-14154) Add explanation for dfs.ha.tail-edits.period in user guide.

2018-12-17 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14154: --- Summary: Add explanation for dfs.ha.tail-edits.period in user guide. Key: HDFS-14154 URL: https://issues.apache.org/jira/browse/HDFS-14154 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-14149) Adjust annotations on new interfaces/classes for SBN reads.

2018-12-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14149: Attachment: HDFS-14149-HDFS-12943.002.patch > Adjust annotations on new interfaces/classes for SBN reads.

[jira] [Commented] (HDFS-14149) Adjust annotations on new interfaces/classes for SBN reads.

2018-12-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723244#comment-16723244 ] Chao Sun commented on HDFS-14149: - Rebase. > Adjust annotations on new interfaces/classes for SBN reads.

[jira] [Commented] (HDFS-14116) ObserverReadProxyProvider should work with protocols other than ClientProtocol

2018-12-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723220#comment-16723220 ] Chao Sun commented on HDFS-14116: - [~shv]: If I understand correctly, the reason to add this in

[jira] [Updated] (HDFS-14149) Adjust annotations on new interfaces/classes for SBN reads.

2018-12-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14149: Attachment: HDFS-14149-HDFS-12943.001.patch > Adjust annotations on new interfaces/classes for SBN reads.

[jira] [Commented] (HDFS-14149) Adjust annotations on new interfaces/classes for SBN reads.

2018-12-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723200#comment-16723200 ] Chao Sun commented on HDFS-14149: - Oops - missed these. Thanks [~shv]. Attached patch v1. > Adjust

[jira] [Assigned] (HDFS-6469) Coordinated replication of the namespace using ConsensusNode

2018-12-14 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HDFS-6469: -- Assignee: Chao Sun (was: Konstantin Shvachko) > Coordinated replication of the namespace using

[jira] [Comment Edited] (HDFS-14146) Handle exception from internalQueueCall

2018-12-14 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721675#comment-16721675 ] Chao Sun edited comment on HDFS-14146 at 12/14/18 6:38 PM: --- Attached patch v3

[jira] [Updated] (HDFS-14146) Handle exception from internalQueueCall

2018-12-14 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14146: Attachment: HDFS-14146-HDFS-12943.003.patch > Handle exception from internalQueueCall >

[jira] [Commented] (HDFS-14146) Handle exception from internalQueueCall

2018-12-14 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721675#comment-16721675 ] Chao Sun commented on HDFS-14146: - Attached patch v3 to address the checkstyle issues as well as removing

[jira] [Commented] (HDFS-14138) Description errors in the comparison logic of transaction ID

2018-12-14 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721667#comment-16721667 ] Chao Sun commented on HDFS-14138: - [~xiangheng]: sure, let me remove that duplication. > Description

[jira] [Updated] (HDFS-14116) ObserverReadProxyProvider should work with protocols other than ClientProtocol

2018-12-13 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14116: Summary: ObserverReadProxyProvider should work with protocols other than ClientProtocol (was: Fix a

[jira] [Commented] (HDFS-14116) Fix a potential class cast error in ObserverReadProxyProvider

2018-12-13 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720949#comment-16720949 ] Chao Sun commented on HDFS-14116: - Thanks [~vagarychen] for verifying the failed tests. Attached patch v4

[jira] [Updated] (HDFS-14149) Adjust annotations on new interfaces/classes for SBN reads.

2018-12-13 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-14149: Status: Patch Available (was: Open) > Adjust annotations on new interfaces/classes for SBN reads. >

<    1   2   3   4   5   6   7   8   9   10   >