[jira] [Updated] (HDFS-12157) Do fsyncDirectory(..) outside of FSDataset lock

2017-07-31 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-12157: - Attachment: HDFS-12157-branch-2.7-01.patch Attaching branch-2.7 patch again for jenkins > Do

[jira] [Updated] (HDFS-12212) Options.Rename.To_TRASH is considered even when Options.Rename.NONE is specified

2017-07-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-12212: - Assignee: Vinayakumar B Status: Patch Available (was: Open) > Options.Rename.To_TRASH is

[jira] [Updated] (HDFS-12212) Options.Rename.To_TRASH is considered even when Options.Rename.NONE is specified

2017-07-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-12212: - Attachment: HDFS-12212-01.patch attached a simple 1-line fix. > Options.Rename.To_TRASH is

[jira] [Created] (HDFS-12212) Options.Rename.To_TRASH is considered even when Options.Rename.NONE is specified

2017-07-28 Thread Vinayakumar B (JIRA)
Vinayakumar B created HDFS-12212: Summary: Options.Rename.To_TRASH is considered even when Options.Rename.NONE is specified Key: HDFS-12212 URL: https://issues.apache.org/jira/browse/HDFS-12212

[jira] [Commented] (HDFS-12120) Use new block for pre-RollingUpgrade files' append requests

2017-07-20 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095805#comment-16095805 ] Vinayakumar B commented on HDFS-12120: -- bq. The variable length block feature is new and has not been

[jira] [Commented] (HDFS-12157) Do fsyncDirectory(..) outside of FSDataset lock

2017-07-20 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095777#comment-16095777 ] Vinayakumar B commented on HDFS-12157: -- Failures seems to be unrelated > Do fsyncDirectory(..)

[jira] [Updated] (HDFS-12157) Do fsyncDirectory(..) outside of FSDataset lock

2017-07-20 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-12157: - Attachment: HDFS-12157-branch-2-01.patch HDFS-12157-branch-2.7-01.patch Attached

[jira] [Updated] (HDFS-12157) Do fsyncDirectory(..) outside of FSDataset lock

2017-07-18 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-12157: - Assignee: Vinayakumar B Target Version/s: 3.0.0-alpha4 Status: Patch

[jira] [Updated] (HDFS-12157) Do fsyncDirectory(..) outside of FSDataset lock

2017-07-18 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-12157: - Attachment: HDFS-12157-01.patch Attached the patch for trunk > Do fsyncDirectory(..) outside of

[jira] [Commented] (HDFS-5042) Completed files lost after power failure

2017-07-18 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091709#comment-16091709 ] Vinayakumar B commented on HDFS-5042: - Created HDFS-12157 for the fsyncDir outside lock. > Completed

[jira] [Updated] (HDFS-12157) Do fsyncDirectory(..) outside of FSDataset lock

2017-07-18 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-12157: - Description: HDFS-5042 introduced fsyncDirectory(..) to save blocks from power failure. Do it

[jira] [Created] (HDFS-12157) Do fsyncDirectory(..) outside of FSDataset lock

2017-07-18 Thread Vinayakumar B (JIRA)
Vinayakumar B created HDFS-12157: Summary: Do fsyncDirectory(..) outside of FSDataset lock Key: HDFS-12157 URL: https://issues.apache.org/jira/browse/HDFS-12157 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-5042) Completed files lost after power failure

2017-07-12 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084300#comment-16084300 ] Vinayakumar B commented on HDFS-5042: - bq. Wondering if we should make this feature configurable This

[jira] [Updated] (HDFS-5042) Completed files lost after power failure

2017-07-11 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-5042: Attachment: HDFS-5042-branch-2.8-addendum.patch Here is the addendum patch to move fsync() out of

[jira] [Commented] (HDFS-5042) Completed files lost after power failure

2017-07-11 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083451#comment-16083451 ] Vinayakumar B commented on HDFS-5042: - Whether doing the fsync() outside the dataset impl lock will

[jira] [Commented] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure

2017-07-11 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082488#comment-16082488 ] Vinayakumar B commented on HDFS-11915: -- Attached patch for branch-2 > Sync rbw dir on the first

[jira] [Updated] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure

2017-07-11 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11915: - Attachment: HDFS-11915-branch-2-01.patch > Sync rbw dir on the first hsync() to avoid file lost

[jira] [Updated] (HDFS-12120) Use new block for pre-RollingUpgrade files` append requests

2017-07-11 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-12120: - Status: Patch Available (was: Open) > Use new block for pre-RollingUpgrade files` append

[jira] [Assigned] (HDFS-12120) Use new block for pre-RollingUpgrade files` append requests

2017-07-11 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B reassigned HDFS-12120: Assignee: Vinayakumar B > Use new block for pre-RollingUpgrade files` append requests >

[jira] [Updated] (HDFS-12120) Use new block for pre-RollingUpgrade files` append requests

2017-07-11 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-12120: - Attachment: HDFS-12120-01.patch Attached the initial patch. Please review. If the approach is

[jira] [Updated] (HDFS-12120) Use new block for pre-RollingUpgrade files' append requests

2017-07-11 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-12120: - Summary: Use new block for pre-RollingUpgrade files' append requests (was: Use new block for

[jira] [Created] (HDFS-12120) Use new block for pre-RollingUpgrade files` append requests

2017-07-11 Thread Vinayakumar B (JIRA)
Vinayakumar B created HDFS-12120: Summary: Use new block for pre-RollingUpgrade files` append requests Key: HDFS-12120 URL: https://issues.apache.org/jira/browse/HDFS-12120 Project: Hadoop HDFS

[jira] [Commented] (HDFS-12120) Use new block for pre-RollingUpgrade files` append requests

2017-07-11 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082172#comment-16082172 ] Vinayakumar B commented on HDFS-12120: -- Idea is : 1. On RU-prepare command, NameNode can track the

[jira] [Updated] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure

2017-07-03 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11915: - Status: Patch Available (was: Open) > Sync rbw dir on the first hsync() to avoid file lost on

[jira] [Updated] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure

2017-07-03 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11915: - Attachment: HDFS-11915-01.patch Attached the patch for trunk. Please review. If this is fine, I

[jira] [Comment Edited] (HDFS-11898) DFSClient#isHedgedReadsEnabled() should be per client flag

2017-07-03 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038195#comment-16038195 ] Vinayakumar B edited comment on HDFS-11898 at 7/3/17 12:03 PM: --- bq. I don't

[jira] [Updated] (HDFS-11738) hedged pread takes more time when block moved from initial locations

2017-06-30 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11738: - Attachment: HDFS-11738-02.patch Updated the patch after HDFS-11708. Please review. > hedged

[jira] [Updated] (HDFS-11067) DFS#listStatusIterator(..) should throw FileNotFoundException if the directory deleted before fetching next batch of entries

2017-06-22 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11067: - Resolution: Fixed Fix Version/s: 3.0.0-alpha4 Release Note:

[jira] [Commented] (HDFS-11067) DFS#listStatusIterator(..) should throw FileNotFoundException if the directory deleted before fetching next batch of entries

2017-06-22 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059227#comment-16059227 ] Vinayakumar B commented on HDFS-11067: -- Thanks [~ajisakaa]. Will add a release note during commit. >

[jira] [Updated] (HDFS-11708) Positional read will fail if replicas moved to different DNs after stream is opened

2017-06-06 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11708: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.2

[jira] [Commented] (HDFS-11708) Positional read will fail if replicas moved to different DNs after stream is opened

2017-06-05 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038221#comment-16038221 ] Vinayakumar B commented on HDFS-11708: -- Jenkins says v07 patch for trunk applies cleanly as well.

[jira] [Updated] (HDFS-11708) Positional read will fail if replicas moved to different DNs after stream is opened

2017-06-05 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11708: - Attachment: HDFS-11708-07.patch Trunk v07 patch seems to apply cleanly for me. Re-attaching the

[jira] [Comment Edited] (HDFS-11898) DFSClient#isHedgedReadsEnabled() should be per client flag

2017-06-05 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038195#comment-16038195 ] Vinayakumar B edited comment on HDFS-11898 at 6/6/17 5:18 AM: -- bq, I don't

[jira] [Commented] (HDFS-11898) DFSClient#isHedgedReadsEnabled() should be per client flag

2017-06-05 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038195#comment-16038195 ] Vinayakumar B commented on HDFS-11898: -- bq, I don't think new variable hedgedReadEnabled is needed.

[jira] [Assigned] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure

2017-06-02 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B reassigned HDFS-11915: Assignee: Vinayakumar B > Sync rbw dir on the first hsync() to avoid file lost on power

[jira] [Updated] (HDFS-11898) DFSClient#isHedgedReadsEnabled() should be per client flag

2017-06-01 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11898: - Attachment: HDFS-11898-02.patch Attached the patch to add the flag per client. May be discussion

[jira] [Commented] (HDFS-5042) Completed files lost after power failure

2017-06-01 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16034160#comment-16034160 ] Vinayakumar B commented on HDFS-5042: - Thanks a lot [~kihwal] for reviews and commit. Thanks everyone

[jira] [Commented] (HDFS-11856) Ability to re-add Upgrading Nodes (remote) to pipeline for future pipeline updates

2017-06-01 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16034156#comment-16034156 ] Vinayakumar B commented on HDFS-11856: -- Thanks a lot [~kihwal] for reviews and commit. > Ability to

[jira] [Commented] (HDFS-11856) Ability to re-add Upgrading Nodes (remote) to pipeline for future pipeline updates

2017-06-01 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16034142#comment-16034142 ] Vinayakumar B commented on HDFS-11856: -- bq. Will this patch help clusters with more than 3 DNs? Yes,

[jira] [Updated] (HDFS-11708) Positional read will fail if replicas moved to different DNs after stream is opened

2017-06-01 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11708: - Attachment: HDFS-11708-branch-2.7-07.patch HDFS-11708-branch-2.8-07.patch

[jira] [Updated] (HDFS-11708) Positional read will fail if replicas moved to different DNs after stream is opened

2017-05-31 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11708: - Attachment: HDFS-11708-07.patch Attached the patch for only this Jira, > Positional read will

[jira] [Updated] (HDFS-5042) Completed files lost after power failure

2017-05-31 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-5042: Attachment: HDFS-5042-branch-2.7-06.patch Updated branch-2.7 patch as well. > Completed files lost

[jira] [Updated] (HDFS-5042) Completed files lost after power failure

2017-05-31 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-5042: Attachment: HDFS-5042-branch-2.8-06.patch Oops. thats the wrong patch I attached. Here is the

[jira] [Updated] (HDFS-5042) Completed files lost after power failure

2017-05-31 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-5042: Attachment: HDFS-5042-branch-2.7-05.patch Attaching branch-2.7 patch as well. > Completed files

[jira] [Updated] (HDFS-5042) Completed files lost after power failure

2017-05-31 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-5042: Attachment: HDFS-5042-branch-2.8-05.patch Attaching the branch-2.8 patch > Completed files lost

[jira] [Commented] (HDFS-11708) Positional read will fail if replicas moved to different DNs after stream is opened

2017-05-30 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16030521#comment-16030521 ] Vinayakumar B commented on HDFS-11708: -- Yes. HDFS-11898 should be committed first. > Positional read

[jira] [Commented] (HDFS-5042) Completed files lost after power failure

2017-05-30 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029798#comment-16029798 ] Vinayakumar B commented on HDFS-5042: - For trunk patch, checkstyle can be ignored, as its inline with

[jira] [Commented] (HDFS-5042) Completed files lost after power failure

2017-05-30 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029796#comment-16029796 ] Vinayakumar B commented on HDFS-5042: - bq. +1 on the syncing rbw on the fisrt hsync(). But let's focus

[jira] [Commented] (HDFS-5042) Completed files lost after power failure

2017-05-30 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028792#comment-16028792 ] Vinayakumar B commented on HDFS-5042: - bq. I think we need to ensure dir sync on hsync() also as client

[jira] [Commented] (HDFS-11898) DFSClient#isHedgedReadsEnabled() should be per client flag

2017-05-29 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028175#comment-16028175 ] Vinayakumar B commented on HDFS-11898: -- bq. Vinayakumar B While the patch 01 LGTM, there seems to be

[jira] [Commented] (HDFS-5042) Completed files lost after power failure

2017-05-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028062#comment-16028062 ] Vinayakumar B commented on HDFS-5042: - Seems some problem with yetus for branch-2. > Completed files

[jira] [Updated] (HDFS-5042) Completed files lost after power failure

2017-05-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-5042: Attachment: HDFS-5042-branch-2-05.patch Re-attaching the branch-2 patch with name changed. >

[jira] [Updated] (HDFS-11889) Datanodes should skip reading replicas from cache during Rollback

2017-05-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11889: - Status: Patch Available (was: Open) > Datanodes should skip reading replicas from cache during

[jira] [Updated] (HDFS-11889) Datanodes should skip reading replicas from cache during Rollback

2017-05-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11889: - Attachment: HDFS-11889-01.patch Attaching the patch. Please review. > Datanodes should skip

[jira] [Updated] (HDFS-5042) Completed files lost after power failure

2017-05-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-5042: Attachment: HDFS-5042-05-branch-2.patch Attaching branch-2 patch > Completed files lost after power

[jira] [Updated] (HDFS-5042) Completed files lost after power failure

2017-05-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-5042: Attachment: HDFS-5042-05.patch Fixed the test failure. > Completed files lost after power failure >

[jira] [Comment Edited] (HDFS-11708) positional read will fail if replicas moved to different DNs after stream is opened

2017-05-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16027764#comment-16027764 ] Vinayakumar B edited comment on HDFS-11708 at 5/28/17 9:57 AM: --- Attached the

[jira] [Updated] (HDFS-11708) positional read will fail if replicas moved to different DNs after stream is opened

2017-05-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11708: - Attachment: HDFS-11708-HDFS-11898-06.patch Attached the patch for the update. 1. Reason for

[jira] [Updated] (HDFS-11898) DFSClient#isHedgedReadsEnabled() should be per client flag

2017-05-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11898: - Status: Patch Available (was: Open) > DFSClient#isHedgedReadsEnabled() should be per client flag

[jira] [Commented] (HDFS-11898) DFSClient#isHedgedReadsEnabled() should be per client flag

2017-05-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16027760#comment-16027760 ] Vinayakumar B commented on HDFS-11898: -- Adding dependency for HDFS-11708, to successful detection of

[jira] [Updated] (HDFS-11898) DFSClient#isHedgedReadsEnabled() should be per client flag

2017-05-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11898: - Attachment: HDFS-11898-01.patch Attached the patch. Found this while trying to run both hedged

[jira] [Created] (HDFS-11898) DFSClient#isHedgedReadsEnabled() should be per client flag

2017-05-28 Thread Vinayakumar B (JIRA)
Vinayakumar B created HDFS-11898: Summary: DFSClient#isHedgedReadsEnabled() should be per client flag Key: HDFS-11898 URL: https://issues.apache.org/jira/browse/HDFS-11898 Project: Hadoop HDFS

[jira] [Assigned] (HDFS-11889) Datanodes should skip reading replicas from cache during Rollback

2017-05-26 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B reassigned HDFS-11889: Assignee: Vinayakumar B Description: During rolling upgrade rollback, Datanodes

[jira] [Created] (HDFS-11889) Datanodes should skip reading replicas from cache during Rollback

2017-05-26 Thread Vinayakumar B (JIRA)
Vinayakumar B created HDFS-11889: Summary: Datanodes should skip reading replicas from cache during Rollback Key: HDFS-11889 URL: https://issues.apache.org/jira/browse/HDFS-11889 Project: Hadoop HDFS

[jira] [Updated] (HDFS-11856) Ability to re-add Upgrading Nodes (remote) to pipeline for future pipeline updates

2017-05-26 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11856: - Attachment: HDFS-11856-branch-2.7-02.patch branch-2.7 patch attached. > Ability to re-add

[jira] [Updated] (HDFS-11856) Ability to re-add Upgrading Nodes (remote) to pipeline for future pipeline updates

2017-05-25 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11856: - Attachment: HDFS-11856-branch-2.8-02.patch patch for branch-2.8 > Ability to re-add Upgrading

[jira] [Updated] (HDFS-11856) Ability to re-add Upgrading Nodes (remote) to pipeline for future pipeline updates

2017-05-25 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11856: - Attachment: HDFS-11856-branch-2-02.patch Attaching patch for branch-2. Others will attach soon.

[jira] [Commented] (HDFS-11445) FSCK shows overall health stauts as corrupt even one replica is corrupt

2017-05-25 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16024464#comment-16024464 ] Vinayakumar B commented on HDFS-11445: -- +1, latest patch looks good to me as well. > FSCK shows

[jira] [Updated] (HDFS-11708) positional read will fail if replicas moved to different DNs after stream is opened

2017-05-24 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11708: - Attachment: HDFS-11708-05.patch Attached the rebased patch.

[jira] [Updated] (HDFS-5042) Completed files lost after power failure

2017-05-24 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-5042: Attachment: HDFS-5042-04.patch Updated the patch > Completed files lost after power failure >

[jira] [Commented] (HDFS-5042) Completed files lost after power failure

2017-05-24 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023356#comment-16023356 ] Vinayakumar B commented on HDFS-5042: - bq. Does it make sense to have FileIoProvider#sync() to call the

[jira] [Commented] (HDFS-5042) Completed files lost after power failure

2017-05-24 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023293#comment-16023293 ] Vinayakumar B commented on HDFS-5042: - bq. fsync(File fileToSync) will leak file descriptors and other

[jira] [Updated] (HDFS-11856) Ability to re-add Upgrading Nodes (remote) to pipeline for future pipeline updates

2017-05-24 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11856: - Attachment: HDFS-11856-02.patch Fixed TestWriteToReplica > Ability to re-add Upgrading Nodes

[jira] [Commented] (HDFS-5042) Completed files lost after power failure

2017-05-24 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022871#comment-16022871 ] Vinayakumar B commented on HDFS-5042: - Hi [~kihwal]/[~andrew.wang] Do changes looks good to you? >

[jira] [Updated] (HDFS-11875) Sort last block's locations for append

2017-05-24 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11875: - Attachment: HDFS-11875-01.patch attached the patch for same > Sort last block's locations for

[jira] [Updated] (HDFS-11875) Sort last block's locations for append

2017-05-24 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11875: - Status: Patch Available (was: Open) > Sort last block's locations for append >

[jira] [Created] (HDFS-11875) Sort last block's locations for append

2017-05-24 Thread Vinayakumar B (JIRA)
Vinayakumar B created HDFS-11875: Summary: Sort last block's locations for append Key: HDFS-11875 URL: https://issues.apache.org/jira/browse/HDFS-11875 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-11856) Ability to re-add Upgrading Nodes (remote) to pipeline for future pipeline updates

2017-05-24 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11856: - Status: Patch Available (was: Open) > Ability to re-add Upgrading Nodes (remote) to pipeline for

[jira] [Updated] (HDFS-11856) Ability to re-add Upgrading Nodes (remote) to pipeline for future pipeline updates

2017-05-24 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11856: - Attachment: HDFS-11856-01.patch Attached patch please review > Ability to re-add Upgrading

[jira] [Commented] (HDFS-5042) Completed files lost after power failure

2017-05-23 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021531#comment-16021531 ] Vinayakumar B commented on HDFS-5042: - Mentioned directory sync will be called on block close() if

[jira] [Assigned] (HDFS-5042) Completed files lost after power failure

2017-05-23 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B reassigned HDFS-5042: --- Assignee: Vinayakumar B > Completed files lost after power failure >

[jira] [Updated] (HDFS-5042) Completed files lost after power failure

2017-05-23 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-5042: Attachment: HDFS-5042-03.patch updated patch > Completed files lost after power failure >

[jira] [Updated] (HDFS-5042) Completed files lost after power failure

2017-05-22 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-5042: Attachment: HDFS-5042-02.patch Updated the patch to fix javadoc and checkstyle > Completed files

[jira] [Commented] (HDFS-11708) positional read will fail if replicas moved to different DNs after stream is opened

2017-05-22 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020641#comment-16020641 ] Vinayakumar B commented on HDFS-11708: -- bq. Not sure I understand what you are trying to fix here. It

[jira] [Updated] (HDFS-5042) Completed files lost after power failure

2017-05-22 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-5042: Status: Patch Available (was: Open) > Completed files lost after power failure >

[jira] [Updated] (HDFS-5042) Completed files lost after power failure

2017-05-22 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-5042: Attachment: HDFS-5042-branch-2-01.patch attached the branch-2 patch > Completed files lost after

[jira] [Updated] (HDFS-5042) Completed files lost after power failure

2017-05-22 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-5042: Attachment: HDFS-5042-01.patch Attached the proposed changes for trunk, > Completed files lost

[jira] [Comment Edited] (HDFS-5042) Completed files lost after power failure

2017-05-19 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017382#comment-16017382 ] Vinayakumar B edited comment on HDFS-5042 at 5/19/17 1:23 PM: -- Thanks

[jira] [Commented] (HDFS-5042) Completed files lost after power failure

2017-05-19 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017382#comment-16017382 ] Vinayakumar B commented on HDFS-5042: - Thanks [~andrew.wang] for the pointer. LUCENE did find the

[jira] [Commented] (HDFS-11856) Ability to re-add Upgrading Nodes (remote) to pipeline for future pipeline updates

2017-05-19 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017316#comment-16017316 ] Vinayakumar B commented on HDFS-11856: -- Solution: 1. Consider the Non-Local restarting nodes as BAD

[jira] [Created] (HDFS-11856) Ability to re-add Upgrading Nodes (remote) to pipeline for future pipeline updates

2017-05-19 Thread Vinayakumar B (JIRA)
Vinayakumar B created HDFS-11856: Summary: Ability to re-add Upgrading Nodes (remote) to pipeline for future pipeline updates Key: HDFS-11856 URL: https://issues.apache.org/jira/browse/HDFS-11856

[jira] [Commented] (HDFS-11849) JournalNode startup failure exception should be logged in log file

2017-05-19 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017305#comment-16017305 ] Vinayakumar B commented on HDFS-11849: -- +1 > JournalNode startup failure exception should be logged

[jira] [Commented] (HDFS-11711) DN should not delete the block On "Too many open files" Exception

2017-05-15 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010411#comment-16010411 ] Vinayakumar B commented on HDFS-11711: -- Changes looks good to me. +1. Also this should be marked as

[jira] [Commented] (HDFS-11738) hedged pread takes more time when block moved from initial locations

2017-05-15 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010399#comment-16010399 ] Vinayakumar B commented on HDFS-11738: -- Hi [~cnauroth]/[~andrew.wang]/[~jingzhao] Can anyone review

[jira] [Updated] (HDFS-11674) reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered

2017-05-12 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11674: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.1

[jira] [Commented] (HDFS-11674) reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered

2017-05-11 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007564#comment-16007564 ] Vinayakumar B commented on HDFS-11674: -- Thanks [~arpitagarwal] for confirmation. Committed to trunk,

[jira] [Updated] (HDFS-11674) reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered

2017-05-11 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11674: - Attachment: HDFS-11674-branch-2.7-03.patch > reserveSpaceForReplicas is not released if append

[jira] [Updated] (HDFS-11674) reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered

2017-05-10 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11674: - Attachment: HDFS-11674-03.patch Attached the updated patch. There was a chance that test could

[jira] [Commented] (HDFS-11674) reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered

2017-05-10 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005886#comment-16005886 ] Vinayakumar B commented on HDFS-11674: -- bq. Could you please clarify how this part works?

[jira] [Issue Comment Deleted] (HDFS-9276) Failed to Update HDFS Delegation Token for long running application in HA mode

2017-05-09 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-9276: Comment: was deleted (was: In long running applications, the tokens will be expired, so we must

<    1   2   3   4   5   6   7   8   9   10   >