[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-20 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387083#comment-15387083 ] Yongjun Zhang commented on HDFS-10587: -- Thanks for your reply [~xupener]. Yes, HDFS

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-20 Thread xupeng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387045#comment-15387045 ] xupeng commented on HDFS-10587: --- Hi [~yzhangal] [~vinayrpet]: Sorry for the late reply, I c

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-19 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383695#comment-15383695 ] Yongjun Zhang commented on HDFS-10587: -- Thanks [~vinayrpet], you are really fast resp

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-19 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383692#comment-15383692 ] Yongjun Zhang commented on HDFS-10587: -- Hi [~xupener], Per [~vinayrpet]'s comment he

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-19 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383690#comment-15383690 ] Vinayakumar B commented on HDFS-10587: -- bq. I did an experiment by reverting HDFS-466

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-19 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383685#comment-15383685 ] Yongjun Zhang commented on HDFS-10587: -- Hi [~vinayrpet], Thanks much again for creat

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-15 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15379864#comment-15379864 ] Yongjun Zhang commented on HDFS-10587: -- Thank you so much [~vinayrpet]! We are lookin

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378879#comment-15378879 ] Vinayakumar B commented on HDFS-10587: -- bq. Here it says checksum error at 81920, whi

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378546#comment-15378546 ] Wei-Chiu Chuang commented on HDFS-10587: I see. So for the logs I saw, there's no

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378483#comment-15378483 ] Yongjun Zhang commented on HDFS-10587: -- I think this looks why we had new data reachi

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15377160#comment-15377160 ] Yongjun Zhang commented on HDFS-10587: -- Thanks a lot [~vinayrpet] and [~xupener]! As

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376940#comment-15376940 ] Vinayakumar B commented on HDFS-10587: -- bq. org.apache.hadoop.fs.ChecksumException: C

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread xupeng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376897#comment-15376897 ] xupeng commented on HDFS-10587: --- hi [~vinayrpet] : logs related are listed below 134.228 {

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376794#comment-15376794 ] Vinayakumar B commented on HDFS-10587: -- Hi [~xupeng], Can you add 229 and 77 logs as

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread xupeng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376750#comment-15376750 ] xupeng commented on HDFS-10587: --- And here are the logs : Hbase log ---

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread xupeng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376739#comment-15376739 ] xupeng commented on HDFS-10587: --- Hi all : I encountered the same issue, here is the scenari

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376576#comment-15376576 ] Vinayakumar B commented on HDFS-10587: -- For End-user both should result in same data.

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376563#comment-15376563 ] Vinayakumar B commented on HDFS-10587: -- bq. I don't see truncate on Sender's, Sender

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376494#comment-15376494 ] Yongjun Zhang commented on HDFS-10587: -- I don't see truncate on Sender's, Sender actu

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376488#comment-15376488 ] Vinayakumar B commented on HDFS-10587: -- >From this, I can see that, truncate happens

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376467#comment-15376467 ] Yongjun Zhang commented on HDFS-10587: -- Hi [~vinayrpet], Thanks for looking into. So

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376464#comment-15376464 ] Wei-Chiu Chuang commented on HDFS-10587: Out of curiosity, Mechanically, what's th

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376462#comment-15376462 ] Wei-Chiu Chuang commented on HDFS-10587: I didn't run hdfs debug verify for other

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376457#comment-15376457 ] Vinayakumar B commented on HDFS-10587: -- bq. From what I understand so far, the corrup

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376455#comment-15376455 ] Wei-Chiu Chuang commented on HDFS-10587: Thanks [~vinayrpet] for the discussion. F

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-14 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376452#comment-15376452 ] Vinayakumar B commented on HDFS-10587: -- bq. Another question is, whats the exact sequ

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-13 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376427#comment-15376427 ] Yongjun Zhang commented on HDFS-10587: -- Thanks [~vinayrpet]. I used "hdfs debug veri

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-13 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376420#comment-15376420 ] Vinayakumar B commented on HDFS-10587: -- bq. In addition, I observed that the data co

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-13 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376414#comment-15376414 ] Yongjun Zhang commented on HDFS-10587: -- Hi [~vinayrpet], Thanks a lot for your comme

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-13 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376368#comment-15376368 ] Vinayakumar B commented on HDFS-10587: -- bq. (5) The last chunk (512 bytes) was not a

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-13 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375922#comment-15375922 ] Yongjun Zhang commented on HDFS-10587: -- The block corruption appears to be corrupted

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-13 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15374480#comment-15374480 ] Yongjun Zhang commented on HDFS-10587: -- About the visibleLength, I saw {code} In Rep

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-12 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373875#comment-15373875 ] Yongjun Zhang commented on HDFS-10587: -- Thanks [~jojochuang] for adding the log to th

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-12 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373705#comment-15373705 ] Yongjun Zhang commented on HDFS-10587: -- Thanks [~jojochuang] for bringing HDFS-4660 t

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-12 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373393#comment-15373393 ] Yongjun Zhang commented on HDFS-10587: -- The failed tests do indicate some issue with

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-12 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372553#comment-15372553 ] Hadoop QA commented on HDFS-10587: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-11 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371798#comment-15371798 ] Yongjun Zhang commented on HDFS-10587: -- HI [~szetszwo] and [~kihwal], I would like t

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-11 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371294#comment-15371294 ] Yongjun Zhang commented on HDFS-10587: -- HI [~jojochuang], I think it'd be nice to wo

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-09 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369328#comment-15369328 ] Yongjun Zhang commented on HDFS-10587: -- I did a quick change {code} diff --git hado

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-08 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15368761#comment-15368761 ] Yongjun Zhang commented on HDFS-10587: -- I see some subtlety here, if the sending DN d

[jira] [Commented] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-08 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15368564#comment-15368564 ] Yongjun Zhang commented on HDFS-10587: -- Hi [~jojochuang], Thanks a lot for the inves