[jira] [Commented] (HDFS-9220) Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum

[ https://issues.apache.org/jira/browse/HDFS-9220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14956328#comment-14956328 ] Jing Zhao commented on HDFS-9220: - The failed tests look like related. Will dig further. >

[jira] [Updated] (HDFS-1172) Blocks in newly completed files are considered under-replicated too quickly

[ https://issues.apache.org/jira/browse/HDFS-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-1172: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Target Version/s: (wa

[jira] [Updated] (HDFS-9223) Code cleanup for DatanodeDescriptor and HeartbeatManager

[ https://issues.apache.org/jira/browse/HDFS-9223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-9223: Attachment: HDFS-9223.003.patch > Code cleanup for DatanodeDescriptor and HeartbeatManager >

[jira] [Updated] (HDFS-9220) Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum

[ https://issues.apache.org/jira/browse/HDFS-9220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-9220: Attachment: HDFS-9220.002.patch Looks like most of the test failures were caused by NULL checksum type. When

[jira] [Commented] (HDFS-8316) Erasure coding: refactor EC constants to be consistent with HDFS-8249

[ https://issues.apache.org/jira/browse/HDFS-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526990#comment-14526990 ] Jing Zhao commented on HDFS-8316: - Thanks for the work, Zhe! +1. > Erasure coding: refacto

[jira] [Resolved] (HDFS-8316) Erasure coding: refactor EC constants to be consistent with HDFS-8249

[ https://issues.apache.org/jira/browse/HDFS-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8316. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed I've committed this to the fe

[jira] [Updated] (HDFS-8281) Erasure Coding: implement parallel stateful reading for striped layout

[ https://issues.apache.org/jira/browse/HDFS-8281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8281: Attachment: HDFS-8281-HDFS-7285.002.patch Thanks again for the comments, Zhe! Update the patch with some code

[jira] [Commented] (HDFS-8203) Erasure Coding: Seek and other Ops in DFSStripedInputStream.

[ https://issues.apache.org/jira/browse/HDFS-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527305#comment-14527305 ] Jing Zhao commented on HDFS-8203: - Thanks for working on this, Yi! Currently we're trying t

[jira] [Resolved] (HDFS-8281) Erasure Coding: implement parallel stateful reading for striped layout

[ https://issues.apache.org/jira/browse/HDFS-8281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8281. - Resolution: Fixed Fix Version/s: HDFS-7285 Thanks again for the review, Zhe and Nicholas! I've commi

[jira] [Created] (HDFS-8319) Erasure Coding: support decoding for stateful read

Jing Zhao created HDFS-8319: --- Summary: Erasure Coding: support decoding for stateful read Key: HDFS-8319 URL: https://issues.apache.org/jira/browse/HDFS-8319 Project: Hadoop HDFS Issue Type: Sub-ta

[jira] [Commented] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527400#comment-14527400 ] Jing Zhao commented on HDFS-8319: - Yeah, sure. I will review HDFS-7678 later today. > Eras

[jira] [Commented] (HDFS-8321) CacheDirectives and CachePool operations should throw RetriableException in safemode

[ https://issues.apache.org/jira/browse/HDFS-8321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527508#comment-14527508 ] Jing Zhao commented on HDFS-8321: - +1 pending Jenkins. > CacheDirectives and CachePool ope

[jira] [Commented] (HDFS-7672) Erasure Coding: handle write failure for stripping coding blocks

[ https://issues.apache.org/jira/browse/HDFS-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527582#comment-14527582 ] Jing Zhao commented on HDFS-7672: - Thanks for working on this, Nicholas! Some comments on t

[jira] [Commented] (HDFS-7678) Erasure coding: DFSInputStream with decode functionality

[ https://issues.apache.org/jira/browse/HDFS-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527716#comment-14527716 ] Jing Zhao commented on HDFS-7678: - Thanks for the great work, Zhe! I have not finished my r

[jira] [Commented] (HDFS-7678) Erasure coding: DFSInputStream with decode functionality (pread)

[ https://issues.apache.org/jira/browse/HDFS-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14528086#comment-14528086 ] Jing Zhao commented on HDFS-7678: - Thanks for the explanation and updating the patch, Zhe!

[jira] [Commented] (HDFS-7672) Erasure Coding: handle write failure for stripping coding blocks

[ https://issues.apache.org/jira/browse/HDFS-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529129#comment-14529129 ] Jing Zhao commented on HDFS-7672: - Thanks for updating the patch, Nicholas! The latest patc

[jira] [Commented] (HDFS-7678) Erasure coding: DFSInputStream with decode functionality (pread)

[ https://issues.apache.org/jira/browse/HDFS-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529685#comment-14529685 ] Jing Zhao commented on HDFS-7678: - Thanks for updating the patch, Zhe! Here are some furthe

[jira] [Commented] (HDFS-6757) Simplify lease manager with INodeID

[ https://issues.apache.org/jira/browse/HDFS-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531562#comment-14531562 ] Jing Zhao commented on HDFS-6757: - Thanks for continuing the work here, Haohui. Some commen

[jira] [Commented] (HDFS-7678) Erasure coding: DFSInputStream with decode functionality (pread)

[ https://issues.apache.org/jira/browse/HDFS-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531620#comment-14531620 ] Jing Zhao commented on HDFS-7678: - Thanks Zhe! bq. So my proposal is that we can use strip

[jira] [Commented] (HDFS-8289) DFSStripedOutputStream uses an additional rpc all to getErasureCodingInfo

[ https://issues.apache.org/jira/browse/HDFS-8289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531675#comment-14531675 ] Jing Zhao commented on HDFS-8289: - Thanks for working on this, Yong! Your patch looks good

[jira] [Commented] (HDFS-8289) DFSStripedOutputStream uses an additional rpc all to getErasureCodingInfo

[ https://issues.apache.org/jira/browse/HDFS-8289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531694#comment-14531694 ] Jing Zhao commented on HDFS-8289: - Ahh, I see why you move ECSchemaProto into hdfs.proto. P

[jira] [Commented] (HDFS-7678) Erasure coding: DFSInputStream with decode functionality (pread)

[ https://issues.apache.org/jira/browse/HDFS-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531703#comment-14531703 ] Jing Zhao commented on HDFS-7678: - bq. To reduce dependency on codec and output stream, I w

[jira] [Commented] (HDFS-8334) Erasure coding: rename DFSStripedInputStream related test classes

[ https://issues.apache.org/jira/browse/HDFS-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531721#comment-14531721 ] Jing Zhao commented on HDFS-8334: - Thanks for working on this, Zhe! One question is, after

[jira] [Commented] (HDFS-7678) Erasure coding: DFSInputStream with decode functionality (pread)

[ https://issues.apache.org/jira/browse/HDFS-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531732#comment-14531732 ] Jing Zhao commented on HDFS-7678: - bq. The content-dismatch failures are pretty hard to deb

[jira] [Commented] (HDFS-8334) Erasure coding: rename DFSStripedInputStream related test classes

[ https://issues.apache.org/jira/browse/HDFS-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531734#comment-14531734 ] Jing Zhao commented on HDFS-8334: - Yeah, this makes sense to me. +1 for the patch. > Erasu

[jira] [Commented] (HDFS-8334) Erasure coding: rename DFSStripedInputStream related test classes

[ https://issues.apache.org/jira/browse/HDFS-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531737#comment-14531737 ] Jing Zhao commented on HDFS-8334: - bq. Also, we have never verified that parity blocks are

[jira] [Commented] (HDFS-8334) Erasure coding: rename DFSStripedInputStream related test classes

[ https://issues.apache.org/jira/browse/HDFS-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531751#comment-14531751 ] Jing Zhao commented on HDFS-8334: - bq. I suppose the +1 still holds Yes! > Erasure coding

[jira] [Commented] (HDFS-8203) Erasure Coding: Seek and other Ops in DFSStripedInputStream.

[ https://issues.apache.org/jira/browse/HDFS-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531770#comment-14531770 ] Jing Zhao commented on HDFS-8203: - Thanks Yi! The patch looks pretty good to me. Some minor

[jira] [Commented] (HDFS-8203) Erasure Coding: Seek and other Ops in DFSStripedInputStream.

[ https://issues.apache.org/jira/browse/HDFS-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533104#comment-14533104 ] Jing Zhao commented on HDFS-8203: - Thanks for updating the patch, Yi! The 003 patch looks p

[jira] [Resolved] (HDFS-8203) Erasure Coding: Seek and other Ops in DFSStripedInputStream.

[ https://issues.apache.org/jira/browse/HDFS-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8203. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed I've committed this to the fe

[jira] [Updated] (HDFS-8289) Erasure Coding: add ECSchema to HdfsFileStatus

[ https://issues.apache.org/jira/browse/HDFS-8289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8289: Summary: Erasure Coding: add ECSchema to HdfsFileStatus (was: DFSStripedOutputStream uses an additional rpc

[jira] [Updated] (HDFS-8289) Erasure Coding: add ECSchema to HdfsFileStatus

[ https://issues.apache.org/jira/browse/HDFS-8289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8289: Attachment: HDFS-8289.004.patch Thanks for updating the patch, Yong. The 003 patch looks good to me except it

[jira] [Resolved] (HDFS-8289) Erasure Coding: add ECSchema to HdfsFileStatus

[ https://issues.apache.org/jira/browse/HDFS-8289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8289. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed I've committed this to the fe

[jira] [Commented] (HDFS-8327) Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate()

[ https://issues.apache.org/jira/browse/HDFS-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535222#comment-14535222 ] Jing Zhao commented on HDFS-8327: - Thanks for working on this code refactoring, Haohui! The

[jira] [Commented] (HDFS-7728) Avoid updating quota usage while loading edits

[ https://issues.apache.org/jira/browse/HDFS-7728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535226#comment-14535226 ] Jing Zhao commented on HDFS-7728: - Sure. Will rebase the patch later. > Avoid updating quo

[jira] [Commented] (HDFS-8355) Erasure Code: Refactor BlockInfo and BlockInfoUnderConstruction

[ https://issues.apache.org/jira/browse/HDFS-8355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535485#comment-14535485 ] Jing Zhao commented on HDFS-8355: - Thanks for the refactoring, Nichoals! The patch looks pr

[jira] [Updated] (HDFS-8355) Erasure Coding: Refactor BlockInfo and BlockInfoUnderConstruction

[ https://issues.apache.org/jira/browse/HDFS-8355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8355: Summary: Erasure Coding: Refactor BlockInfo and BlockInfoUnderConstruction (was: Erasure Code: Refactor Bloc

[jira] [Resolved] (HDFS-8355) Erasure Coding: Refactor BlockInfo and BlockInfoUnderConstruction

[ https://issues.apache.org/jira/browse/HDFS-8355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8355. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed I've committed this. > Erasu

[jira] [Commented] (HDFS-6757) Simplify lease manager with INodeID

[ https://issues.apache.org/jira/browse/HDFS-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535793#comment-14535793 ] Jing Zhao commented on HDFS-6757: - The 016 patch looks pretty good to me. But the IOExcepti

[jira] [Commented] (HDFS-8327) Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate()

[ https://issues.apache.org/jira/browse/HDFS-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535837#comment-14535837 ] Jing Zhao commented on HDFS-8327: - The 004 patch looks good to me. +1 pending Jenkins. > C

[jira] [Commented] (HDFS-7678) Erasure coding: DFSInputStream with decode functionality (pread)

[ https://issues.apache.org/jira/browse/HDFS-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14537493#comment-14537493 ] Jing Zhao commented on HDFS-7678: - Thanks for the great work, Zhe! The 013 patch looks pret

[jira] [Commented] (HDFS-7678) Erasure coding: DFSInputStream with decode functionality (pread)

[ https://issues.apache.org/jira/browse/HDFS-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539068#comment-14539068 ] Jing Zhao commented on HDFS-7678: - Thanks for updating the patch, Zhe! +1 for the 014 patch

[jira] [Updated] (HDFS-8372) Erasure coding: compute storage type quotas for striped files, to be consistent with HDFS-8327

[ https://issues.apache.org/jira/browse/HDFS-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8372: Attachment: HDFS-8372-HDFS-7285.1.patch Thanks for taking care of this, Zhe! The .0 patch looks good to me. I

[jira] [Updated] (HDFS-8372) Erasure coding: compute storage type quotas for striped files, to be consistent with HDFS-8327

[ https://issues.apache.org/jira/browse/HDFS-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8372: Resolution: Pending Closed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed Status: Reso

[jira] [Reopened] (HDFS-8372) Erasure coding: compute storage type quotas for striped files, to be consistent with HDFS-8327

[ https://issues.apache.org/jira/browse/HDFS-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reopened HDFS-8372: - > Erasure coding: compute storage type quotas for striped files, to be > consistent with HDFS-8327 > -

[jira] [Resolved] (HDFS-8372) Erasure coding: compute storage type quotas for striped files, to be consistent with HDFS-8327

[ https://issues.apache.org/jira/browse/HDFS-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8372. - Resolution: Fixed > Erasure coding: compute storage type quotas for striped files, to be > consistent with

[jira] [Updated] (HDFS-7728) Avoid updating quota usage while loading edits

[ https://issues.apache.org/jira/browse/HDFS-7728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7728: Attachment: HDFS-7728.002.patch Rebase the patch: add QuotaDelta into ReclaimContext. > Avoid updating quota

[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface

[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541080#comment-14541080 ] Jing Zhao commented on HDFS-8345: - Thanks for working on this, Arpit! The patch looks good

[jira] [Commented] (HDFS-8347) Erasure Coding: whether to use the same chunkSize in decoding with the value in encoding

[ https://issues.apache.org/jira/browse/HDFS-8347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541100#comment-14541100 ] Jing Zhao commented on HDFS-8347: - For now (HDFS-7285) can we just hardcode {{cellSize}} an

[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface

[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14542305#comment-14542305 ] Jing Zhao commented on HDFS-8345: - Looks like most of the public functions in {{BlockStorag

[jira] [Commented] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14542403#comment-14542403 ] Jing Zhao commented on HDFS-8319: - Thanks Zhe! I will include the refactoring here. > Eras

[jira] [Commented] (HDFS-8323) Bump GenerationStamp for write faliure in DFSStripedOutputStream

[ https://issues.apache.org/jira/browse/HDFS-8323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14542451#comment-14542451 ] Jing Zhao commented on HDFS-8323: - Thanks for working on this, Nicholas! The patch looks go

[jira] [Updated] (HDFS-7728) Avoid updating quota usage while loading edits

[ https://issues.apache.org/jira/browse/HDFS-7728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7728: Attachment: HDFS-7728.003.patch Update the patch to address comments. > Avoid updating quota usage while loa

[jira] [Commented] (HDFS-8398) Erasure Coding: Correctly calculate last striped block length in DFSStripedInputStream if it's under construction.

[ https://issues.apache.org/jira/browse/HDFS-8398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14543231#comment-14543231 ] Jing Zhao commented on HDFS-8398: - HDFS-7661 aims to support reading a being written file.

[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface

[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14544933#comment-14544933 ] Jing Zhao commented on HDFS-8345: - Thanks for updating the patch, Arpit! The latest patch l

[jira] [Commented] (HDFS-8397) Refactor the error handling code in DataStreamer

[ https://issues.apache.org/jira/browse/HDFS-8397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14545886#comment-14545886 ] Jing Zhao commented on HDFS-8397: - Thanks for the refactoring, Nicholas! The patch looks pr

[jira] [Commented] (HDFS-8394) Move getAdditionalBlock() and related functionalities into a separate class

[ https://issues.apache.org/jira/browse/HDFS-8394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14545967#comment-14545967 ] Jing Zhao commented on HDFS-8394: - Thanks for working on this, Haohui. The patch looks good

[jira] [Updated] (HDFS-8397) Refactor the error handling code in DataStreamer

[ https://issues.apache.org/jira/browse/HDFS-8397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8397: Attachment: HDFS-8397.002.patch Since the suggested change is trivial, I just updated the patch for Nicholas.

[jira] [Commented] (HDFS-8394) Move getAdditionalBlock() and related functionalities into a separate class

[ https://issues.apache.org/jira/browse/HDFS-8394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546001#comment-14546001 ] Jing Zhao commented on HDFS-8394: - The 005 patch looks good to me. +1 pending Jenkins. > M

[jira] [Commented] (HDFS-8345) Storage policy APIs must be exposed via the FileSystem interface

[ https://issues.apache.org/jira/browse/HDFS-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546187#comment-14546187 ] Jing Zhao commented on HDFS-8345: - bq. Is there a reason not to include a convenience metho

[jira] [Commented] (HDFS-8320) Erasure coding: consolidate striping-related terminologies

[ https://issues.apache.org/jira/browse/HDFS-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546251#comment-14546251 ] Jing Zhao commented on HDFS-8320: - Thanks for working on this, Zhe! The patch looks pretty

[jira] [Work started] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-8319 started by Jing Zhao. --- > Erasure Coding: support decoding for stateful read > -

[jira] [Updated] (HDFS-8397) Refactor the error handling code in DataStreamer

[ https://issues.apache.org/jira/browse/HDFS-8397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8397: Resolution: Fixed Fix Version/s: 2.8.0 Hadoop Flags: Reviewed Status: Resolved (was: P

[jira] [Commented] (HDFS-8367) BlockInfoStriped uses EC schema

[ https://issues.apache.org/jira/browse/HDFS-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549195#comment-14549195 ] Jing Zhao commented on HDFS-8367: - Thanks [~kaisasak] for working on this and [~drankye] fo

[jira] [Commented] (HDFS-8323) Bump GenerationStamp for write faliure in DFSStripedOutputStream

[ https://issues.apache.org/jira/browse/HDFS-8323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549260#comment-14549260 ] Jing Zhao commented on HDFS-8323: - Thanks for updating the patch, Nicholas! Checking the pa

[jira] [Commented] (HDFS-8418) Fix the isNeededReplication calculation for Striped block in NN

[ https://issues.apache.org/jira/browse/HDFS-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549284#comment-14549284 ] Jing Zhao commented on HDFS-8418: - Thanks for the fix, Yi! The patch looks pretty good to m

[jira] [Updated] (HDFS-8417) Erasure Coding: Pread failed to read data starting from not-first stripe

[ https://issues.apache.org/jira/browse/HDFS-8417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8417: Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed Status: Resolved (wa

[jira] [Commented] (HDFS-8367) BlockInfoStriped uses EC schema

[ https://issues.apache.org/jira/browse/HDFS-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549360#comment-14549360 ] Jing Zhao commented on HDFS-8367: - Yeah, maybe we can do it in a new jira since this will b

[jira] [Commented] (HDFS-8320) Erasure coding: consolidate striping-related terminologies

[ https://issues.apache.org/jira/browse/HDFS-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549656#comment-14549656 ] Jing Zhao commented on HDFS-8320: - Thanks for updating the patch, Zhe! The new patch looks

[jira] [Commented] (HDFS-8418) Fix the isNeededReplication calculation for Striped block in NN

[ https://issues.apache.org/jira/browse/HDFS-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549658#comment-14549658 ] Jing Zhao commented on HDFS-8418: - bq. INodeFile#getPreferredBlockReplication is method of

[jira] [Updated] (HDFS-8418) Fix the isNeededReplication calculation for Striped block in NN

[ https://issues.apache.org/jira/browse/HDFS-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8418: Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed Status: Resolved (wa

[jira] [Commented] (HDFS-7687) Change fsck to support EC files

[ https://issues.apache.org/jira/browse/HDFS-7687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549798#comment-14549798 ] Jing Zhao commented on HDFS-7687: - FYI, I've merged HDFS-8405 into the feature branch. > C

[jira] [Updated] (HDFS-8320) Erasure coding: consolidate striping-related terminologies

[ https://issues.apache.org/jira/browse/HDFS-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8320: Attachment: HDFS-8320-HDFS-7285.03.patch Thanks Zhe! +1 for the 02 patch after addressing the following nits:

[jira] [Commented] (HDFS-7991) Allow users to skip checkpoint when stopping NameNode

[ https://issues.apache.org/jira/browse/HDFS-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551234#comment-14551234 ] Jing Zhao commented on HDFS-7991: - Recently we just saw several clusters from our customers

[jira] [Resolved] (HDFS-4942) Add retry cache support in Namenode

[ https://issues.apache.org/jira/browse/HDFS-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-4942. - Resolution: Fixed Fix Version/s: 2.1.0-beta > Add retry cache support in Namenode >

[jira] [Resolved] (HDFS-8323) Bump GenerationStamp for write faliure in DFSStripedOutputStream

[ https://issues.apache.org/jira/browse/HDFS-8323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8323. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed +1 on the latest patch. I've

[jira] [Updated] (HDFS-8427) Remove dataBlockNum and parityBlockNum from BlockInfoStriped

[ https://issues.apache.org/jira/browse/HDFS-8427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8427: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) +1. I've co

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14553292#comment-14553292 ] Jing Zhao commented on HDFS-7609: - Thanks for working on this, [~mingma]! Your analysis mak

[jira] [Commented] (HDFS-8421) Move startFile() and related operations into FSDirWriteFileOp

[ https://issues.apache.org/jira/browse/HDFS-8421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14553416#comment-14553416 ] Jing Zhao commented on HDFS-8421: - # We do not need to remove the following check from the

[jira] [Commented] (HDFS-8421) Move startFile() and related operations into FSDirWriteFileOp

[ https://issues.apache.org/jira/browse/HDFS-8421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14553421#comment-14553421 ] Jing Zhao commented on HDFS-8421: - +1 after addressing the comments. > Move startFile() an

[jira] [Commented] (HDFS-7991) Allow users to skip checkpoint when stopping NameNode

[ https://issues.apache.org/jira/browse/HDFS-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14554800#comment-14554800 ] Jing Zhao commented on HDFS-7991: - Thanks Allen. Yes, I also just realized that jmx may not

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555024#comment-14555024 ] Jing Zhao commented on HDFS-7609: - Thanks for the response, Ming! Yes I agree that in most

[jira] [Commented] (HDFS-7991) Allow users to skip checkpoint when stopping NameNode

[ https://issues.apache.org/jira/browse/HDFS-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555038#comment-14555038 ] Jing Zhao commented on HDFS-7991: - Thanks for the further explanation, Allen. Now I get you

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555402#comment-14555402 ] Jing Zhao commented on HDFS-7609: - Spent some further time digging into the issue. Besides

[jira] [Updated] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8319: Attachment: HDFS-8319.001.patch Initial patch. The main change is to unify the stateful read code and the pre

[jira] [Commented] (HDFS-8319) Erasure Coding: support decoding for stateful read

[ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14559508#comment-14559508 ] Jing Zhao commented on HDFS-8319: - Thanks for the review, [~walter.k.su]. Replied your comm

[jira] [Commented] (HDFS-7991) Allow users to skip checkpoint when stopping NameNode

[ https://issues.apache.org/jira/browse/HDFS-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14559813#comment-14559813 ] Jing Zhao commented on HDFS-7991: - We can use this jira just to remove the original dfsadmi

[jira] [Commented] (HDFS-8479) Erasure coding: fix striping related logic in FSDirWriteFileOp to sync with HDFS-8421

[ https://issues.apache.org/jira/browse/HDFS-8479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14560067#comment-14560067 ] Jing Zhao commented on HDFS-8479: - Thanks for the merge, Zhe! The patch looks pretty good t

[jira] [Commented] (HDFS-8479) Erasure coding: fix striping related logic in FSDirWriteFileOp to sync with HDFS-8421

[ https://issues.apache.org/jira/browse/HDFS-8479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14560084#comment-14560084 ] Jing Zhao commented on HDFS-8479: - BTW, while running tests after the merge, I got this tes

[jira] [Commented] (HDFS-8479) Erasure coding: fix striping related logic in FSDirWriteFileOp to sync with HDFS-8421

[ https://issues.apache.org/jira/browse/HDFS-8479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14560097#comment-14560097 ] Jing Zhao commented on HDFS-8479: - Looks like the failure is caused by HADOOP-11847. I will

[jira] [Updated] (HDFS-8479) Erasure coding: fix striping related logic in FSDirWriteFileOp to sync with HDFS-8421

[ https://issues.apache.org/jira/browse/HDFS-8479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8479: Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed Status: Resolved (wa

[jira] [Commented] (HDFS-8479) Erasure coding: fix striping related logic in FSDirWriteFileOp to sync with HDFS-8421

[ https://issues.apache.org/jira/browse/HDFS-8479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14560114#comment-14560114 ] Jing Zhao commented on HDFS-8479: - Maybe we can open a new jira to include this change and

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14561354#comment-14561354 ] Jing Zhao commented on HDFS-7609: - Thanks Ming! The new patch looks good to me. One minor i

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14561372#comment-14561372 ] Jing Zhao commented on HDFS-7609: - bq. any ideas why the current implementation sets NN to

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14562543#comment-14562543 ] Jing Zhao commented on HDFS-7609: - Thanks for sharing the thoughts, Ming! Totally agree wit

[jira] [Commented] (HDFS-8460) Erasure Coding: stateful read result doesn't match data occasionally because of flawed test

[ https://issues.apache.org/jira/browse/HDFS-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14565129#comment-14565129 ] Jing Zhao commented on HDFS-8460: - We can use {{DataNodeTestUtil#setHeartbeatsDisabledForTe

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14565161#comment-14565161 ] Jing Zhao commented on HDFS-7609: - The 03 patch looks good to me. +1. I will commit it shor

[jira] [Updated] (HDFS-7609) Avoid retry cache collision when Standby NameNode loading edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7609: Summary: Avoid retry cache collision when Standby NameNode loading edits (was: startup used too much time to

[jira] [Updated] (HDFS-7609) Avoid retry cache collision when Standby NameNode loading edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7609: Resolution: Fixed Fix Version/s: 2.8.0 Hadoop Flags: Reviewed Status: Resolved (was: P

[jira] [Updated] (HDFS-7609) Avoid retry cache collision when Standby NameNode loading edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7609: Issue Type: Bug (was: Improvement) > Avoid retry cache collision when Standby NameNode loading edits > -

<    1   2   3   4   5   6   7   8   9   10   >