[jira] [Updated] (HDFS-7609) Avoid retry cache collision when Standby NameNode loading edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7609: Priority: Critical (was: Major) > Avoid retry cache collision when Standby NameNode loading edits >

[jira] [Updated] (HDFS-7609) Avoid retry cache collision when Standby NameNode loading edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7609: Labels: (was: BB2015-05-RFC) > Avoid retry cache collision when Standby NameNode loading edits > --

[jira] [Commented] (HDFS-8481) Erasure coding: remove workarounds in client side stripped blocks recovering

[ https://issues.apache.org/jira/browse/HDFS-8481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14565224#comment-14565224 ] Jing Zhao commented on HDFS-8481: - Thanks for working on this, Zhe! I agree that we should

[jira] [Commented] (HDFS-8481) Erasure coding: remove workarounds in client side stripped blocks recovering

[ https://issues.apache.org/jira/browse/HDFS-8481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14565447#comment-14565447 ] Jing Zhao commented on HDFS-8481: - Yes, to have this kind of accumulation will be great. Bu

[jira] [Commented] (HDFS-8489) Subclass BlockInfo to represent contiguous blocks

[ https://issues.apache.org/jira/browse/HDFS-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14565556#comment-14565556 ] Jing Zhao commented on HDFS-8489: - Thanks for working on this, Zhe. Yes, addStorage, remove

[jira] [Commented] (HDFS-8489) Subclass BlockInfo to represent contiguous blocks

[ https://issues.apache.org/jira/browse/HDFS-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14567662#comment-14567662 ] Jing Zhao commented on HDFS-8489: - The 04 patch looks pretty good to me. One nitty pick is

[jira] [Commented] (HDFS-8270) create() always retried with hardcoded timeout when file already exists

[ https://issues.apache.org/jira/browse/HDFS-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14567684#comment-14567684 ] Jing Zhao commented on HDFS-8270: - Totally agree this retry logic should not be hardcoded i

[jira] [Commented] (HDFS-8489) Subclass BlockInfo to represent contiguous blocks

[ https://issues.apache.org/jira/browse/HDFS-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14567780#comment-14567780 ] Jing Zhao commented on HDFS-8489: - +1 for the 05 patch. Since the only change between 04 an

[jira] [Updated] (HDFS-8489) Subclass BlockInfo to represent contiguous blocks

[ https://issues.apache.org/jira/browse/HDFS-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8489: Resolution: Fixed Fix Version/s: 2.8.0 Hadoop Flags: Reviewed Status: Resolved (was: P

[jira] [Updated] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8319: Attachment: HDFS-8319.002.patch Update the patch that # addresses Walter's comments # Update the original {{f

[jira] [Commented] (HDFS-8481) Erasure coding: remove workarounds in client side stripped blocks recovering

[ https://issues.apache.org/jira/browse/HDFS-8481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568680#comment-14568680 ] Jing Zhao commented on HDFS-8481: - While debugging HDFS-8319, I just found that in {{TestW

[jira] [Commented] (HDFS-8453) Erasure coding: properly handle start offset for internal blocks in a block group

[ https://issues.apache.org/jira/browse/HDFS-8453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568756#comment-14568756 ] Jing Zhao commented on HDFS-8453: - bq. So this patch takes another approach: refactor DFSIn

[jira] [Commented] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569471#comment-14569471 ] Jing Zhao commented on HDFS-8319: - Thanks for the comments, Walter! Actually this has been

[jira] [Commented] (HDFS-8517) Fix a decoding issue in stripped block recovering in client side

[ https://issues.apache.org/jira/browse/HDFS-8517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569798#comment-14569798 ] Jing Zhao commented on HDFS-8517: - Thanks for the fix, Kai! The patch looks good to me. Jus

[jira] [Updated] (HDFS-8517) Fix a decoding issue in stripped block recovering in client side

[ https://issues.apache.org/jira/browse/HDFS-8517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8517: Attachment: HDFS-8517-HDFS-7285.v2.patch Since the remaining issues are mostly minor, I just made these chang

[jira] [Commented] (HDFS-8517) Fix a decoding issue in stripped block recovering in client side

[ https://issues.apache.org/jira/browse/HDFS-8517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569903#comment-14569903 ] Jing Zhao commented on HDFS-8517: - Thanks for the quick review, Zhe and Kai! I will commit

[jira] [Updated] (HDFS-8517) Fix a decoding issue in stripped block recovering in client side

[ https://issues.apache.org/jira/browse/HDFS-8517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8517: Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed Status: Resolved (wa

[jira] [Commented] (HDFS-8453) Erasure coding: properly handle start offset for internal blocks in a block group

[ https://issues.apache.org/jira/browse/HDFS-8453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569924#comment-14569924 ] Jing Zhao commented on HDFS-8453: - The patch looks pretty good to me. +1 after making the t

[jira] [Commented] (HDFS-8453) Erasure coding: properly handle start offset for internal blocks in a block group

[ https://issues.apache.org/jira/browse/HDFS-8453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569955#comment-14569955 ] Jing Zhao commented on HDFS-8453: - +1 for the 01 patch. > Erasure coding: properly handle

[jira] [Updated] (HDFS-8453) Erasure coding: properly handle start offset for internal blocks in a block group

[ https://issues.apache.org/jira/browse/HDFS-8453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8453: Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed Status: Resolved (wa

[jira] [Commented] (HDFS-8270) create() always retried with hardcoded timeout when file already exists

[ https://issues.apache.org/jira/browse/HDFS-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14570362#comment-14570362 ] Jing Zhao commented on HDFS-8270: - bq. In that case I don't think removing would be problem

[jira] [Updated] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8319: Attachment: HDFS-8319.003.patch Thanks a lot for the review, guys! Update the patch to address comments. I wi

[jira] [Commented] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571900#comment-14571900 ] Jing Zhao commented on HDFS-8319: - bq. Using the same buffer type (no mixing) will also mak

[jira] [Comment Edited] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571900#comment-14571900 ] Jing Zhao edited comment on HDFS-8319 at 6/4/15 12:36 AM: -- bq. Usi

[jira] [Commented] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571929#comment-14571929 ] Jing Zhao commented on HDFS-8319: - bq. So looks like the inputOffsets doesn't work? The in

[jira] [Commented] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571991#comment-14571991 ] Jing Zhao commented on HDFS-8319: - bq. We could also not use slice call, instead use duplic

[jira] [Commented] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14572017#comment-14572017 ] Jing Zhao commented on HDFS-8319: - bq. By the way, I noticed you have some code refactoring

[jira] [Updated] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8319: Attachment: HDFS-8319.004.patch The 004 patch removes coder change and uses direct buffer for parity chunks.

[jira] [Commented] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14572095#comment-14572095 ] Jing Zhao commented on HDFS-8319: - bq. For pread, initDecodeInputs does the job of preparin

[jira] [Updated] (HDFS-8531) Append failed due to unreleased lease from previous appender with quota exceeded exception

[ https://issues.apache.org/jira/browse/HDFS-8531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8531: Reporter: Sumana Sathish (was: Xiaoyu Yao) > Append failed due to unreleased lease from previous appender wi

[jira] [Commented] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573821#comment-14573821 ] Jing Zhao commented on HDFS-8319: - Thanks a lot for the review, Zhe! bq. Is it possible th

[jira] [Updated] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8319: Attachment: HDFS-8319.005.patch Update the patch to address Zhe's comments. > Erasure Coding: support decodi

[jira] [Commented] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573843#comment-14573843 ] Jing Zhao commented on HDFS-8319: - but we only read one more parity block each time we hit

[jira] [Commented] (HDFS-8494) Remove hard-coded chunk size in favor of ECZone

[ https://issues.apache.org/jira/browse/HDFS-8494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577586#comment-14577586 ] Jing Zhao commented on HDFS-8494: - Thanks for working on this, [~kaisasak]! In general I'm

[jira] [Commented] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577605#comment-14577605 ] Jing Zhao commented on HDFS-7827: - I just found that for a legacy fsimage we currently alwa

[jira] [Created] (HDFS-8559) Erasure Coding: fix non-protobuf fsimage for striped blocks

Jing Zhao created HDFS-8559: --- Summary: Erasure Coding: fix non-protobuf fsimage for striped blocks Key: HDFS-8559 URL: https://issues.apache.org/jira/browse/HDFS-8559 Project: Hadoop HDFS Issue Ty

[jira] [Commented] (HDFS-8494) Remove hard-coded chunk size in favor of ECZone

[ https://issues.apache.org/jira/browse/HDFS-8494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577615#comment-14577615 ] Jing Zhao commented on HDFS-8494: - Also, we should consider adding a chunk size field to {{

[jira] [Updated] (HDFS-8559) Erasure Coding: fix non-protobuf fsimage for striped blocks

[ https://issues.apache.org/jira/browse/HDFS-8559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8559: Attachment: HDFS-8559.000.patch For now we can save and load striped blocks just as contiguous blocks in a l

[jira] [Commented] (HDFS-8561) checkNNStartup should throw a RetriableException

[ https://issues.apache.org/jira/browse/HDFS-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14579705#comment-14579705 ] Jing Zhao commented on HDFS-8561: - Thanks for working on this, Arpit! The v2 patch looks go

[jira] [Commented] (HDFS-8587) Erasure Coding: fix the copy constructor of BlockInfoStriped and BlockInfoContiguous

[ https://issues.apache.org/jira/browse/HDFS-8587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14583879#comment-14583879 ] Jing Zhao commented on HDFS-8587: - Thanks for the fix, Yi! The patch looks good to me. One

[jira] [Commented] (HDFS-8585) Erasure Coding: Remove dataBlockNum and parityBlockNum from StripedBlockProto

[ https://issues.apache.org/jira/browse/HDFS-8585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584137#comment-14584137 ] Jing Zhao commented on HDFS-8585: - Thanks for working on this, Yi! +1. I will commit the pa

[jira] [Updated] (HDFS-8585) Erasure Coding: Remove dataBlockNum and parityBlockNum from StripedBlockProto

[ https://issues.apache.org/jira/browse/HDFS-8585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8585: Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed Status: Resolved (wa

[jira] [Commented] (HDFS-8580) Erasure coding: Persist cellSize in BlockInfoStriped and StripedBlockProto

[ https://issues.apache.org/jira/browse/HDFS-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584164#comment-14584164 ] Jing Zhao commented on HDFS-8580: - actually StripedBlockProto is only used in fsimage befor

[jira] [Commented] (HDFS-8593) Calculation of effective layout version mishandles comparison to current layout version in storage.

[ https://issues.apache.org/jira/browse/HDFS-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584729#comment-14584729 ] Jing Zhao commented on HDFS-8593: - Thanks for the fix, Chris! The patch looks pretty good t

[jira] [Commented] (HDFS-8580) Erasure coding: Persist cellSize in BlockInfoStriped and StripedBlocksFeature

[ https://issues.apache.org/jira/browse/HDFS-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14586398#comment-14586398 ] Jing Zhao commented on HDFS-8580: - The 002 patch looks good to me. One comment is that whil

[jira] [Commented] (HDFS-7912) Erasure Coding: track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks

[ https://issues.apache.org/jira/browse/HDFS-7912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14586404#comment-14586404 ] Jing Zhao commented on HDFS-7912: - Yeah, that will be helpful. > Erasure Coding: track Blo

[jira] [Commented] (HDFS-8543) Erasure Coding: processOverReplicatedBlock() handles striped block

[ https://issues.apache.org/jira/browse/HDFS-8543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587106#comment-14587106 ] Jing Zhao commented on HDFS-8543: - Thanks for working on this, Walter! So some comments oth

[jira] [Commented] (HDFS-8361) Choose SSD over DISK in block placement

[ https://issues.apache.org/jira/browse/HDFS-8361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587144#comment-14587144 ] Jing Zhao commented on HDFS-8361: - Thanks for working on this, Nicholas! It may be easier t

[jira] [Commented] (HDFS-8592) SafeModeException never get unwrapped

[ https://issues.apache.org/jira/browse/HDFS-8592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587485#comment-14587485 ] Jing Zhao commented on HDFS-8592: - +1 > SafeModeException never get unwrapped > --

[jira] [Commented] (HDFS-8580) Erasure coding: Persist cellSize in BlockInfoStriped and StripedBlocksFeature

[ https://issues.apache.org/jira/browse/HDFS-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588530#comment-14588530 ] Jing Zhao commented on HDFS-8580: - # Let's use "optional" here to avoid incompatibility whe

[jira] [Commented] (HDFS-8608) Merge HDFS-7912 to trunk (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks)

[ https://issues.apache.org/jira/browse/HDFS-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588718#comment-14588718 ] Jing Zhao commented on HDFS-8608: - Thanks for working on this, Zhe! The patch looks pretty

[jira] [Commented] (HDFS-8446) Separate safemode related operations in GetBlockLocations()

[ https://issues.apache.org/jira/browse/HDFS-8446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588964#comment-14588964 ] Jing Zhao commented on HDFS-8446: - # Printing out the file paths can still be useful here.

[jira] [Updated] (HDFS-8580) Erasure coding: Persist cellSize in BlockInfoStriped and StripedBlocksFeature

[ https://issues.apache.org/jira/browse/HDFS-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8580: Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed Status: Resolved (wa

[jira] [Commented] (HDFS-8446) Separate safemode related operations in GetBlockLocations()

[ https://issues.apache.org/jira/browse/HDFS-8446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590296#comment-14590296 ] Jing Zhao commented on HDFS-8446: - +1 for the 003 patch. > Separate safemode related opera

[jira] [Updated] (HDFS-8602) Erasure Coding: Client can't read(decode) the EC files which have corrupt blocks.

[ https://issues.apache.org/jira/browse/HDFS-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8602: Attachment: HDFS-8602.000.patch Thanks very much for reporting the issue and working on this, [~kaisasak]! I

[jira] [Created] (HDFS-8619) Erasure Coding: revisit replica counting for striped blocks

Jing Zhao created HDFS-8619: --- Summary: Erasure Coding: revisit replica counting for striped blocks Key: HDFS-8619 URL: https://issues.apache.org/jira/browse/HDFS-8619 Project: Hadoop HDFS Issue Ty

[jira] [Updated] (HDFS-8254) Avoid assigning a leading streamer in StripedDataStreamer to tolerate datanode failure

[ https://issues.apache.org/jira/browse/HDFS-8254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8254: Summary: Avoid assigning a leading streamer in StripedDataStreamer to tolerate datanode failure (was: In Str

[jira] [Resolved] (HDFS-8254) Avoid assigning a leading streamer in StripedDataStreamer to tolerate datanode failure

[ https://issues.apache.org/jira/browse/HDFS-8254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8254. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed +1 as well. I've committed th

[jira] [Commented] (HDFS-8543) Erasure Coding: processOverReplicatedBlock() handles striped block

[ https://issues.apache.org/jira/browse/HDFS-8543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14593775#comment-14593775 ] Jing Zhao commented on HDFS-8543: - The test failures are caused by "java.lang.NoClassDefFou

[jira] [Updated] (HDFS-8543) Erasure Coding: processOverReplicatedBlock() handles striped block

[ https://issues.apache.org/jira/browse/HDFS-8543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8543: Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed Status: Resolved (wa

[jira] [Resolved] (HDFS-8602) Erasure Coding: Client can't read(decode) the EC files which have corrupt blocks.

[ https://issues.apache.org/jira/browse/HDFS-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8602. - Resolution: Fixed Hadoop Flags: Reviewed Thanks for the confirmation and updating the patch, [~kaisas

[jira] [Commented] (HDFS-8619) Erasure Coding: revisit replica counting for striped blocks

[ https://issues.apache.org/jira/browse/HDFS-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594201#comment-14594201 ] Jing Zhao commented on HDFS-8619: - The scenario mentioned in the description has been fixed

[jira] [Updated] (HDFS-8619) Erasure Coding: revisit replica counting for striped blocks

[ https://issues.apache.org/jira/browse/HDFS-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8619: Attachment: HDFS-8619.000.patch Initial patch to fix the above issue. > Erasure Coding: revisit replica coun

[jira] [Commented] (HDFS-8619) Erasure Coding: revisit replica counting for striped blocks

[ https://issues.apache.org/jira/browse/HDFS-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596905#comment-14596905 ] Jing Zhao commented on HDFS-8619: - Thanks for the review, Zhe! Sure, I will separate some r

[jira] [Commented] (HDFS-8619) Erasure Coding: revisit replica counting for striped blocks

[ https://issues.apache.org/jira/browse/HDFS-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596959#comment-14596959 ] Jing Zhao commented on HDFS-8619: - No. I guess we still need this jira for adding the strip

[jira] [Commented] (HDFS-8619) Erasure Coding: revisit replica counting for striped blocks

[ https://issues.apache.org/jira/browse/HDFS-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596967#comment-14596967 ] Jing Zhao commented on HDFS-8619: - Besides, we now have merged quite a few changes to trunk

[jira] [Commented] (HDFS-8619) Erasure Coding: revisit replica counting for striped blocks

[ https://issues.apache.org/jira/browse/HDFS-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14597211#comment-14597211 ] Jing Zhao commented on HDFS-8619: - bq. Hopefully also to split it to functional pieces like

[jira] [Created] (HDFS-8652) Track BlockInfo instead of Block in CorruptReplicasMap

Jing Zhao created HDFS-8652: --- Summary: Track BlockInfo instead of Block in CorruptReplicasMap Key: HDFS-8652 URL: https://issues.apache.org/jira/browse/HDFS-8652 Project: Hadoop HDFS Issue Type: Im

[jira] [Updated] (HDFS-8652) Track BlockInfo instead of Block in CorruptReplicasMap

[ https://issues.apache.org/jira/browse/HDFS-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8652: Status: Patch Available (was: Open) > Track BlockInfo instead of Block in CorruptReplicasMap > -

[jira] [Updated] (HDFS-8652) Track BlockInfo instead of Block in CorruptReplicasMap

[ https://issues.apache.org/jira/browse/HDFS-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8652: Attachment: HDFS-8652.000.patch > Track BlockInfo instead of Block in CorruptReplicasMap > --

[jira] [Commented] (HDFS-8619) Erasure Coding: revisit replica counting for striped blocks

[ https://issues.apache.org/jira/browse/HDFS-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14598086#comment-14598086 ] Jing Zhao commented on HDFS-8619: - Thanks for the review, Nicholas and Walter! I've created

[jira] [Commented] (HDFS-7749) Erasure Coding: Add striped block support in INodeFile

[ https://issues.apache.org/jira/browse/HDFS-7749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14598151#comment-14598151 ] Jing Zhao commented on HDFS-7749: - Looks like they are the same. So please feel free to cha

[jira] [Commented] (HDFS-8567) Erasure Coding: SafeMode handles file smaller than a full stripe

[ https://issues.apache.org/jira/browse/HDFS-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14598332#comment-14598332 ] Jing Zhao commented on HDFS-8567: - The 01 patch looks good to me. For the failed test TestS

[jira] [Commented] (HDFS-8639) Option for HTTP port of NameNode by MiniDFSClusterManager

[ https://issues.apache.org/jira/browse/HDFS-8639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14598881#comment-14598881 ] Jing Zhao commented on HDFS-8639: - Thanks for working on this, [~kaisasak]! The patch looks

[jira] [Updated] (HDFS-8639) Option for HTTP port of NameNode by MiniDFSClusterManager

[ https://issues.apache.org/jira/browse/HDFS-8639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8639: Resolution: Fixed Fix Version/s: 2.8.0 Hadoop Flags: Reviewed Status: Resolved (was: P

[jira] [Updated] (HDFS-8639) Option for HTTP port of NameNode by MiniDFSClusterManager

[ https://issues.apache.org/jira/browse/HDFS-8639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8639: Priority: Minor (was: Trivial) > Option for HTTP port of NameNode by MiniDFSClusterManager > ---

[jira] [Updated] (HDFS-8567) Erasure Coding: SafeMode handles file smaller than a full stripe

[ https://issues.apache.org/jira/browse/HDFS-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8567: Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed Status: Resolved (wa

[jira] [Commented] (HDFS-8623) Refactor NameNode handling of invalid, corrupt, and under-recovery blocks

[ https://issues.apache.org/jira/browse/HDFS-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600163#comment-14600163 ] Jing Zhao commented on HDFS-8623: - Thanks for working on this, Zhe. The patch looks good to

[jira] [Commented] (HDFS-8655) Refactor accesses to INodeFile#blocks

[ https://issues.apache.org/jira/browse/HDFS-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600199#comment-14600199 ] Jing Zhao commented on HDFS-8655: - Thanks for working on this, Zhe. One thing I agree with

[jira] [Commented] (HDFS-8623) Refactor NameNode handling of invalid, corrupt, and under-recovery blocks

[ https://issues.apache.org/jira/browse/HDFS-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600226#comment-14600226 ] Jing Zhao commented on HDFS-8623: - +1 pending Jenkins. > Refactor NameNode handling of inv

[jira] [Commented] (HDFS-8655) Refactor accesses to INodeFile#blocks

[ https://issues.apache.org/jira/browse/HDFS-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600228#comment-14600228 ] Jing Zhao commented on HDFS-8655: - bq. Should we also rename blocks to contiguousBlocks we

[jira] [Commented] (HDFS-7285) Erasure Coding Support inside HDFS

[ https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600384#comment-14600384 ] Jing Zhao commented on HDFS-7285: - bq. With the new BlockInfo hierarchy introduced in HDFS-

[jira] [Updated] (HDFS-8666) speedup TestMover

[ https://issues.apache.org/jira/browse/HDFS-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8666: Status: Patch Available (was: Open) > speedup TestMover > - > > Key: HDFS-86

[jira] [Commented] (HDFS-8483) Erasure coding: test DataNode reporting bad/corrupted blocks which belongs to a striped block.

[ https://issues.apache.org/jira/browse/HDFS-8483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600609#comment-14600609 ] Jing Zhao commented on HDFS-8483: - Yeah, actually we can add more tests in HDFS-8619 to ver

[jira] [Commented] (HDFS-8623) Refactor NameNode handling of invalid, corrupt, and under-recovery blocks

[ https://issues.apache.org/jira/browse/HDFS-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602033#comment-14602033 ] Jing Zhao commented on HDFS-8623: - No, we do not need to wait for Jenkins again. The findbu

[jira] [Created] (HDFS-8669) Erasure Coding: handle missing internal block locations in DFSStripedInputStream

Jing Zhao created HDFS-8669: --- Summary: Erasure Coding: handle missing internal block locations in DFSStripedInputStream Key: HDFS-8669 URL: https://issues.apache.org/jira/browse/HDFS-8669 Project: Hadoop HD

[jira] [Updated] (HDFS-8669) Erasure Coding: handle missing internal block locations in DFSStripedInputStream

[ https://issues.apache.org/jira/browse/HDFS-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8669: Description: Currently DFSStripedInputStream assumes we always have complete internal block location informat

[jira] [Updated] (HDFS-8669) Erasure Coding: handle missing internal block locations in DFSStripedInputStream

[ https://issues.apache.org/jira/browse/HDFS-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8669: Attachment: HDFS-8669.000.patch Initial patch to fix. The main change is to add capability in StripeReader so

[jira] [Updated] (HDFS-8623) Refactor NameNode handling of invalid, corrupt, and under-recovery blocks

[ https://issues.apache.org/jira/browse/HDFS-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8623: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was: P

[jira] [Commented] (HDFS-8447) Decouple information of files in GetLocatedBlocks

[ https://issues.apache.org/jira/browse/HDFS-8447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603676#comment-14603676 ] Jing Zhao commented on HDFS-8447: - # Let's add a comment here explaining the intention of t

[jira] [Updated] (HDFS-8652) Track BlockInfo instead of Block in CorruptReplicasMap

[ https://issues.apache.org/jira/browse/HDFS-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8652: Attachment: HDFS-8652.001.patch Rebase the patch after HDFS-8623 got committed. Also use this chance to do so

[jira] [Updated] (HDFS-8666) speedup TestMover

[ https://issues.apache.org/jira/browse/HDFS-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8666: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was: P

[jira] [Commented] (HDFS-8652) Track BlockInfo instead of Block in CorruptReplicasMap

[ https://issues.apache.org/jira/browse/HDFS-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14615674#comment-14615674 ] Jing Zhao commented on HDFS-8652: - Looks like the test TestFileTruncate#testTruncateWithDa

[jira] [Updated] (HDFS-8652) Track BlockInfo instead of Block in CorruptReplicasMap

[ https://issues.apache.org/jira/browse/HDFS-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8652: Attachment: HDFS-8652.002.patch Did some debugging and confirmed that the failure of TestFIleTruncate#testTr

[jira] [Updated] (HDFS-8652) Track BlockInfo instead of Block in CorruptReplicasMap

[ https://issues.apache.org/jira/browse/HDFS-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8652: Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch Available) I've commit

[jira] [Commented] (HDFS-8461) Erasure coding: fix priority level of UnderReplicatedBlocks for striped block

[ https://issues.apache.org/jira/browse/HDFS-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14615893#comment-14615893 ] Jing Zhao commented on HDFS-8461: - Thanks for working on this, [~walter.k.su]! The patch lo

[jira] [Updated] (HDFS-8461) Erasure coding: fix priority level of UnderReplicatedBlocks for striped block

[ https://issues.apache.org/jira/browse/HDFS-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8461: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 Status: Resolved (wa

[jira] [Commented] (HDFS-8652) Track BlockInfo instead of Block in CorruptReplicasMap

[ https://issues.apache.org/jira/browse/HDFS-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14617017#comment-14617017 ] Jing Zhao commented on HDFS-8652: - bq. A BlockInfo's genstamp and/or size may mutate. This

[jira] [Commented] (HDFS-8723) Integrate the build infrastructure with hdfs-client

[ https://issues.apache.org/jira/browse/HDFS-8723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14617340#comment-14617340 ] Jing Zhao commented on HDFS-8723: - +1 > Integrate the build infrastructure with hdfs-clien

[jira] [Commented] (HDFS-8724) Import third_party libraries into the repository

[ https://issues.apache.org/jira/browse/HDFS-8724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14617353#comment-14617353 ] Jing Zhao commented on HDFS-8724: - +1 > Import third_party libraries into the repository >

<    2   3   4   5   6   7   8   9   10   11   >