[jira] [Updated] (HDFS-8873) Allow the directoryScanner to be rate-limited

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8873: --- Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch A

[jira] [Commented] (HDFS-9087) Add some jitter to DataNode.checkDiskErrorThread

[ https://issues.apache.org/jira/browse/HDFS-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14909207#comment-14909207 ] Colin Patrick McCabe commented on HDFS-9087: +1. Thanks, [~eclark]. > Add som

[jira] [Commented] (HDFS-8873) Allow the directoryScanner to be rate-limited

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14909206#comment-14909206 ] Colin Patrick McCabe commented on HDFS-8873: +1. Thanks, [~templedf]. The tes

[jira] [Updated] (HDFS-8873) Allow the directoryScanner to be rate-limited

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8873: --- Summary: Allow the directoryScanner to be rate-limited (was: throttle directoryScanner

[jira] [Updated] (HDFS-9107) Prevent NN's unrecoverable death spiral after full GC

[ https://issues.apache.org/jira/browse/HDFS-9107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9107: --- Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch A

[jira] [Commented] (HDFS-9107) Prevent NN's unrecoverable death spiral after full GC

[ https://issues.apache.org/jira/browse/HDFS-9107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14908763#comment-14908763 ] Colin Patrick McCabe commented on HDFS-9107: +1. Test failures not related. W

[jira] [Commented] (HDFS-9133) ExternalBlockReader and ReplicaAccessor need to return -1 on read when at EOF

[ https://issues.apache.org/jira/browse/HDFS-9133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14908727#comment-14908727 ] Colin Patrick McCabe commented on HDFS-9133: I filed HDFS-9147 to fix the namin

[jira] [Created] (HDFS-9147) ExternalBlockReader should not treat bytesRemaining as visibleLength

Colin Patrick McCabe created HDFS-9147: -- Summary: ExternalBlockReader should not treat bytesRemaining as visibleLength Key: HDFS-9147 URL: https://issues.apache.org/jira/browse/HDFS-9147 Project:

[jira] [Commented] (HDFS-9133) ExternalBlockReader and ReplicaAccessor need to return -1 on read when at EOF

[ https://issues.apache.org/jira/browse/HDFS-9133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14908720#comment-14908720 ] Colin Patrick McCabe commented on HDFS-9133: bq. Thanks Colin for the work. The

[jira] [Commented] (HDFS-9132) Pass genstamp to ReplicaAccessorBuilder

[ https://issues.apache.org/jira/browse/HDFS-9132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14908708#comment-14908708 ] Colin Patrick McCabe commented on HDFS-9132: The ReplicaAccessor may be used to

[jira] [Commented] (HDFS-8873) throttle directoryScanner

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14908707#comment-14908707 ] Colin Patrick McCabe commented on HDFS-8873: Another repeat of the jenkins "noc

[jira] [Updated] (HDFS-9132) Pass genstamp to ReplicaAccessorBuilder

[ https://issues.apache.org/jira/browse/HDFS-9132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9132: --- Status: Patch Available (was: Open) > Pass genstamp to ReplicaAccessorBuilder > --

[jira] [Updated] (HDFS-9132) Pass genstamp to ReplicaAccessorBuilder

[ https://issues.apache.org/jira/browse/HDFS-9132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9132: --- Attachment: HDFS-9132.001.patch > Pass genstamp to ReplicaAccessorBuilder > ---

[jira] [Updated] (HDFS-9132) Pass genstamp to ReplicaAccessorBuilder

[ https://issues.apache.org/jira/browse/HDFS-9132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9132: --- Summary: Pass genstamp to ReplicaAccessorBuilder (was: Pass genstamp to ExternalReplic

[jira] [Updated] (HDFS-9133) ExternalBlockReader and ReplicaAccessor need to return -1 on read when at EOF

[ https://issues.apache.org/jira/browse/HDFS-9133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9133: --- Status: Patch Available (was: Open) > ExternalBlockReader and ReplicaAccessor need to

[jira] [Updated] (HDFS-9133) ExternalBlockReader and ReplicaAccessor need to return -1 on read when at EOF

[ https://issues.apache.org/jira/browse/HDFS-9133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9133: --- Attachment: HDFS-9133.001.patch > ExternalBlockReader and ReplicaAccessor need to retur

[jira] [Created] (HDFS-9133) ExternalBlockReader and ReplicaAccessor need to return -1 on read when at EOF

Colin Patrick McCabe created HDFS-9133: -- Summary: ExternalBlockReader and ReplicaAccessor need to return -1 on read when at EOF Key: HDFS-9133 URL: https://issues.apache.org/jira/browse/HDFS-9133

[jira] [Created] (HDFS-9132) Pass genstamp to ExternalReplicaBuilder

Colin Patrick McCabe created HDFS-9132: -- Summary: Pass genstamp to ExternalReplicaBuilder Key: HDFS-9132 URL: https://issues.apache.org/jira/browse/HDFS-9132 Project: Hadoop HDFS Issue T

[jira] [Comment Edited] (HDFS-8873) throttle directoryScanner

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14905122#comment-14905122 ] Colin Patrick McCabe edited comment on HDFS-8873 at 9/23/15 7:47 PM:

[jira] [Commented] (HDFS-8873) throttle directoryScanner

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14905122#comment-14905122 ] Colin Patrick McCabe commented on HDFS-8873: I guess the short summary is that

[jira] [Updated] (HDFS-9080) update htrace version to 4.0

[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9080: --- Attachment: HDFS-9080.010.patch address latest comments > update htrace version to 4.0

[jira] [Commented] (HDFS-9080) update htrace version to 4.0

[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903662#comment-14903662 ] Colin Patrick McCabe commented on HDFS-9080: Thanks for the reviews. bq. Sugge

[jira] [Commented] (HDFS-8873) throttle directoryScanner

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903507#comment-14903507 ] Colin Patrick McCabe commented on HDFS-8873: bq. I didn't do that in this patch

[jira] [Commented] (HDFS-8873) throttle directoryScanner

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903220#comment-14903220 ] Colin Patrick McCabe commented on HDFS-8873: Thanks, [~templedf]. {code} 416

[jira] [Commented] (HDFS-9107) Prevent NN's unrecoverable death spiral after full GC

[ https://issues.apache.org/jira/browse/HDFS-9107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903077#comment-14903077 ] Colin Patrick McCabe commented on HDFS-9107: Also (although I don't feel strong

[jira] [Commented] (HDFS-9107) Prevent NN's unrecoverable death spiral after full GC

[ https://issues.apache.org/jira/browse/HDFS-9107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903048#comment-14903048 ] Colin Patrick McCabe commented on HDFS-9107: I guess if we want to be 100% corr

[jira] [Commented] (HDFS-9122) DN automatically add more volumes to avoid large volume

[ https://issues.apache.org/jira/browse/HDFS-9122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14902920#comment-14902920 ] Colin Patrick McCabe commented on HDFS-9122: It is an interesting idea, but I t

[jira] [Commented] (HDFS-9105) separate replication config for client to create a file or set replication

[ https://issues.apache.org/jira/browse/HDFS-9105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14902894#comment-14902894 ] Colin Patrick McCabe commented on HDFS-9105: I think it makes more sense to use

[jira] [Commented] (HDFS-9087) Add some jitter to DataNode.checkDiskErrorThread

[ https://issues.apache.org/jira/browse/HDFS-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14902879#comment-14902879 ] Colin Patrick McCabe commented on HDFS-9087: Thanks for this, [~eclark]. So, t

[jira] [Commented] (HDFS-8873) throttle directoryScanner

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14901285#comment-14901285 ] Colin Patrick McCabe commented on HDFS-8873: [~nroberts], I agree that it might

[jira] [Commented] (HDFS-9107) Prevent NN's unrecoverable death spiral after full GC

[ https://issues.apache.org/jira/browse/HDFS-9107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14901165#comment-14901165 ] Colin Patrick McCabe commented on HDFS-9107: bq. I don't trust monotonicNow if

[jira] [Updated] (HDFS-9080) update htrace version to 4.0

[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9080: --- Attachment: HDFS-9080.009.patch > update htrace version to 4.0 > --

[jira] [Updated] (HDFS-9080) update htrace version to 4.0

[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9080: --- Attachment: HDFS-9080.007.patch > update htrace version to 4.0 > --

[jira] [Commented] (HDFS-9080) update htrace version to 4.0

[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14876247#comment-14876247 ] Colin Patrick McCabe commented on HDFS-9080: * fix a case where some tests were

[jira] [Commented] (HDFS-8873) throttle directoryScanner

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14876099#comment-14876099 ] Colin Patrick McCabe commented on HDFS-8873: bq. Question about the rest of you

[jira] [Commented] (HDFS-8873) throttle directoryScanner

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14804846#comment-14804846 ] Colin Patrick McCabe commented on HDFS-8873: Thanks, [~templedf]. Looking bett

[jira] [Updated] (HDFS-9080) update htrace version to 4.0

[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9080: --- Attachment: HDFS-9080.006.patch fix compile issue > update htrace version to 4.0 > ---

[jira] [Updated] (HDFS-9080) update htrace version to 4.0

[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9080: --- Attachment: HDFS-9080.005.patch > update htrace version to 4.0 > --

[jira] [Commented] (HDFS-9080) update htrace version to 4.0

[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14804720#comment-14804720 ] Colin Patrick McCabe commented on HDFS-9080: Thanks for the review. bq. You wa

[jira] [Updated] (HDFS-9080) update htrace version to 4.0

[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9080: --- Attachment: HDFS-9080.004.patch > update htrace version to 4.0 > --

[jira] [Commented] (HDFS-8873) throttle directoryScanner

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14804503#comment-14804503 ] Colin Patrick McCabe commented on HDFS-8873: The jenkins errors look like: {cod

[jira] [Commented] (HDFS-9047) deprecate libwebhdfs in branch-2; remove from trunk

[ https://issues.apache.org/jira/browse/HDFS-9047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14804319#comment-14804319 ] Colin Patrick McCabe commented on HDFS-9047: Like I said, I don't have any obje

[jira] [Commented] (HDFS-7492) If multiple threads call FsVolumeList#checkDirs at the same time, we should only do checkDirs once and give the results to all waiting threads

[ https://issues.apache.org/jira/browse/HDFS-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14803419#comment-14803419 ] Colin Patrick McCabe commented on HDFS-7492: [~eclark], also check HDFS-8845 fo

[jira] [Updated] (HDFS-9080) update htrace version to 4.0

[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9080: --- Attachment: HDFS-9080.003.patch rebase on trunk > update htrace version to 4.0 > -

[jira] [Commented] (HDFS-8961) Investigate lock issue in o.a.h.hdfs.shortcircuit.DfsClientShmManager.EndpointShmManager

[ https://issues.apache.org/jira/browse/HDFS-8961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746138#comment-14746138 ] Colin Patrick McCabe commented on HDFS-8961: Thank you for fixing the findbugs

[jira] [Updated] (HDFS-9080) update htrace version to 4.0

[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9080: --- Attachment: HDFS-9080.002.patch > update htrace version to 4.0 > --

[jira] [Commented] (HDFS-9055) WebHDFS REST v2

[ https://issues.apache.org/jira/browse/HDFS-9055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745872#comment-14745872 ] Colin Patrick McCabe commented on HDFS-9055: [~aw], can you explain what you fi

[jira] [Commented] (HDFS-9047) deprecate libwebhdfs in branch-2; remove from trunk

[ https://issues.apache.org/jira/browse/HDFS-9047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745854#comment-14745854 ] Colin Patrick McCabe commented on HDFS-9047: [~aw], if you think "some random c

[jira] [Updated] (HDFS-9080) update htrace version to 4.0

[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9080: --- Attachment: HDFS-9080.001.patch > update htrace version to 4.0 > --

[jira] [Updated] (HDFS-9080) update htrace version to 4.0

[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9080: --- Status: Patch Available (was: Open) > update htrace version to 4.0 > -

[jira] [Created] (HDFS-9080) update htrace version to 4.0

Colin Patrick McCabe created HDFS-9080: -- Summary: update htrace version to 4.0 Key: HDFS-9080 URL: https://issues.apache.org/jira/browse/HDFS-9080 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-8873) throttle directoryScanner

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14744628#comment-14744628 ] Colin Patrick McCabe commented on HDFS-8873: bq. Encapsulating the throttle stu

[jira] [Commented] (HDFS-9051) webhdfs should support recursive list

[ https://issues.apache.org/jira/browse/HDFS-9051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14744510#comment-14744510 ] Colin Patrick McCabe commented on HDFS-9051: I think it's problematic to move m

[jira] [Commented] (HDFS-9058) enable find via WebHDFS

[ https://issues.apache.org/jira/browse/HDFS-9058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14744509#comment-14744509 ] Colin Patrick McCabe commented on HDFS-9058: I think it's problematic to move m

[jira] [Commented] (HDFS-9055) WebHDFS REST v2

[ https://issues.apache.org/jira/browse/HDFS-9055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14744506#comment-14744506 ] Colin Patrick McCabe commented on HDFS-9055: A big part of the goal of webhdfs

[jira] [Comment Edited] (HDFS-9011) Support splitting BlockReport of a storage into multiple RPC

[ https://issues.apache.org/jira/browse/HDFS-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14744469#comment-14744469 ] Colin Patrick McCabe edited comment on HDFS-9011 at 9/14/15 11:10 PM: ---

[jira] [Commented] (HDFS-9011) Support splitting BlockReport of a storage into multiple RPC

[ https://issues.apache.org/jira/browse/HDFS-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14744469#comment-14744469 ] Colin Patrick McCabe commented on HDFS-9011: You can just raise the maximum RPC

[jira] [Commented] (HDFS-8829) Make SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol sockets and allow configuring auto-tuning

[ https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14744458#comment-14744458 ] Colin Patrick McCabe commented on HDFS-8829: +1. Committed to 2.8. Failing te

[jira] [Updated] (HDFS-8829) Make SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol sockets and allow configuring auto-tuning

[ https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8829: --- Resolution: Fixed Fix Version/s: 2.8.0 Target Version/s: 2.8.0

[jira] [Updated] (HDFS-8829) Make SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol sockets and allow configuring auto-tuning

[ https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8829: --- Summary: Make SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol socket

[jira] [Updated] (HDFS-8829) Make SO_RCVBUF size configurable for DataTransferProtocol sockets and allow configuring auto-tuning

[ https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8829: --- Summary: Make SO_RCVBUF size configurable for DataTransferProtocol sockets and allow co

[jira] [Updated] (HDFS-8996) Consolidate validateLog and scanLog in FJM#EditLogFile

[ https://issues.apache.org/jira/browse/HDFS-8996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8996: --- Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch A

[jira] [Commented] (HDFS-8996) Consolidate validateLog and scanLog in FJM#EditLogFile

[ https://issues.apache.org/jira/browse/HDFS-8996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14744431#comment-14744431 ] Colin Patrick McCabe commented on HDFS-8996: Yes, the JN needs to ignore the la

[jira] [Commented] (HDFS-9047) deprecate libwebhdfs in branch-2; remove from trunk

[ https://issues.apache.org/jira/browse/HDFS-9047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14744247#comment-14744247 ] Colin Patrick McCabe commented on HDFS-9047: libwebhdfs fills a purpose that no

[jira] [Commented] (HDFS-8873) throttle directoryScanner

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14741622#comment-14741622 ] Colin Patrick McCabe commented on HDFS-8873: Good work, [~templedf]. {code}

[jira] [Comment Edited] (HDFS-8829) DataNode sets SO_RCVBUF explicitly is disabling tcp auto-tuning

[ https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14741447#comment-14741447 ] Colin Patrick McCabe edited comment on HDFS-8829 at 9/11/15 7:47 PM:

[jira] [Commented] (HDFS-8829) DataNode sets SO_RCVBUF explicitly is disabling tcp auto-tuning

[ https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14741447#comment-14741447 ] Colin Patrick McCabe commented on HDFS-8829: Thanks for working on this, [~He T

[jira] [Commented] (HDFS-8996) Consolidate validateLog and scanLog in FJM#EditLogFile

[ https://issues.apache.org/jira/browse/HDFS-8996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14740082#comment-14740082 ] Colin Patrick McCabe commented on HDFS-8996: +1. Note that we are now using sc

[jira] [Commented] (HDFS-9005) Provide support for upgrade domain script

[ https://issues.apache.org/jira/browse/HDFS-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739386#comment-14739386 ] Colin Patrick McCabe commented on HDFS-9005: Yes, a refresh command would be ni

[jira] [Updated] (HDFS-8860) Remove unused Replica copyOnWrite code

[ https://issues.apache.org/jira/browse/HDFS-8860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8860: --- Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch A

[jira] [Updated] (HDFS-8860) Remove unused Replica copyOnWrite code

[ https://issues.apache.org/jira/browse/HDFS-8860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8860: --- Summary: Remove unused Replica copyOnWrite code (was: Remove Replica hardlink / unlink

[jira] [Commented] (HDFS-8860) Remove Replica hardlink / unlink code

[ https://issues.apache.org/jira/browse/HDFS-8860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14735768#comment-14735768 ] Colin Patrick McCabe commented on HDFS-8860: It looks like this code was origin

[jira] [Commented] (HDFS-9005) Provide support for upgrade domain script

[ https://issues.apache.org/jira/browse/HDFS-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14735519#comment-14735519 ] Colin Patrick McCabe commented on HDFS-9005: With the script approach, how does

[jira] [Commented] (HDFS-9005) Provide support for upgrade domain script

[ https://issues.apache.org/jira/browse/HDFS-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14735356#comment-14735356 ] Colin Patrick McCabe commented on HDFS-9005: Hi [~mingma], Why not use an XML

[jira] [Commented] (HDFS-9021) Use a yellow elephant rather than a blue one in diagram

[ https://issues.apache.org/jira/browse/HDFS-9021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729837#comment-14729837 ] Colin Patrick McCabe commented on HDFS-9021: Thanks, Andrew. It is good to see

[jira] [Updated] (HDFS-8964) When validating the edit log, do not read at or beyond the file offset that is being written

[ https://issues.apache.org/jira/browse/HDFS-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8964: --- Resolution: Fixed Fix Version/s: 2.8 Status: Resolved (was: Patch Ava

[jira] [Updated] (HDFS-8964) When validating the edit log, do not read at or beyond the file offset that is being written

[ https://issues.apache.org/jira/browse/HDFS-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8964: --- Description: NN/JN validates in-progress edit log files in multiple scenarios, via {{E

[jira] [Updated] (HDFS-8964) When validating the edit log, do not read at or beyond the file offset that is being written

[ https://issues.apache.org/jira/browse/HDFS-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8964: --- Summary: When validating the edit log, do not read at or beyond the file offset that is

[jira] [Updated] (HDFS-8964) INotify should not read at or beyond the file offset that is being written in FileJournalManager

[ https://issues.apache.org/jira/browse/HDFS-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8964: --- Summary: INotify should not read at or beyond the file offset that is being written in

[jira] [Commented] (HDFS-8964) Provide max TxId when validating in-progress edit log files

[ https://issues.apache.org/jira/browse/HDFS-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729506#comment-14729506 ] Colin Patrick McCabe commented on HDFS-8964: +1. Thanks, [~zhz]. > Provide ma

[jira] [Commented] (HDFS-8989) Allow rate-limiting the DirectoryScanner via time-slicing

[ https://issues.apache.org/jira/browse/HDFS-8989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728394#comment-14728394 ] Colin Patrick McCabe commented on HDFS-8989: Yeah, I didn't see HDFS-8873 when

[jira] [Commented] (HDFS-8944) Make dfsadmin command options case insensitive

[ https://issues.apache.org/jira/browse/HDFS-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728391#comment-14728391 ] Colin Patrick McCabe commented on HDFS-8944: This seems inconsistent with other

[jira] [Commented] (HDFS-8964) Provide max TxId when validating in-progress edit log files

[ https://issues.apache.org/jira/browse/HDFS-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727967#comment-14727967 ] Colin Patrick McCabe commented on HDFS-8964: {{FileJournalManager}}: it seems l

[jira] [Updated] (HDFS-2070) Add more unit tests for FsShell getmerge

[ https://issues.apache.org/jira/browse/HDFS-2070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-2070: --- Resolution: Fixed Fix Version/s: 2.8 Target Version/s: 2.8

[jira] [Updated] (HDFS-2070) Add more unit tests for FsShell getmerge

[ https://issues.apache.org/jira/browse/HDFS-2070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-2070: --- Summary: Add more unit tests for FsShell getmerge (was: A lack of auto-test for FsShel

[jira] [Commented] (HDFS-2070) A lack of auto-test for FsShell getmerge in hdfs

[ https://issues.apache.org/jira/browse/HDFS-2070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727893#comment-14727893 ] Colin Patrick McCabe commented on HDFS-2070: +1. Thanks, [~templedf]. > A lac

[jira] [Commented] (HDFS-8940) Support for large-scale multi-tenant inotify service

[ https://issues.apache.org/jira/browse/HDFS-8940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14726465#comment-14726465 ] Colin Patrick McCabe commented on HDFS-8940: Thanks for the explanation, [~ming

[jira] [Resolved] (HDFS-9000) Allow reconfiguration without restart for parameters where applicable.

[ https://issues.apache.org/jira/browse/HDFS-9000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe resolved HDFS-9000. Resolution: Duplicate Duplicate of HDFS-1477 and HDFS-7106 > Allow reconfiguration w

[jira] [Updated] (HDFS-8965) Harden edit log reading code against out of memory errors

[ https://issues.apache.org/jira/browse/HDFS-8965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8965: --- Resolution: Fixed Fix Version/s: 2.8 Status: Resolved (was: Patch Ava

[jira] [Commented] (HDFS-8965) Harden edit log reading code against out of memory errors

[ https://issues.apache.org/jira/browse/HDFS-8965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14724514#comment-14724514 ] Colin Patrick McCabe commented on HDFS-8965: TestDNFencingWithReplication and b

[jira] [Commented] (HDFS-8982) Consolidate getFileReplication and getPreferredBlockReplication in INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14724059#comment-14724059 ] Colin Patrick McCabe commented on HDFS-8982: [~wheat9]: it seems to me that it

[jira] [Commented] (HDFS-5995) TestFSEditLogLoader#testValidateEditLogWithCorruptBody gets OutOfMemoryError and dumps heap.

[ https://issues.apache.org/jira/browse/HDFS-5995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14724022#comment-14724022 ] Colin Patrick McCabe commented on HDFS-5995: Update: So, it turns out that the

[jira] [Commented] (HDFS-8985) Restarted namenode suffer from block report storm

[ https://issues.apache.org/jira/browse/HDFS-8985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14724012#comment-14724012 ] Colin Patrick McCabe commented on HDFS-8985: Seems like a duplicate of HDFS-792

[jira] [Updated] (HDFS-8965) Harden edit log reading code against out of memory errors

[ https://issues.apache.org/jira/browse/HDFS-8965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8965: --- Attachment: HDFS-8965.007.patch missed a skip() > Harden edit log reading code against

[jira] [Commented] (HDFS-8964) Provide max TxId when validating in-progress edit log files

[ https://issues.apache.org/jira/browse/HDFS-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14723950#comment-14723950 ] Colin Patrick McCabe commented on HDFS-8964: Thank you, [~zhz]. JNStorage: {{i

[jira] [Updated] (HDFS-8965) Harden edit log reading code against out of memory errors

[ https://issues.apache.org/jira/browse/HDFS-8965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8965: --- Attachment: HDFS-8965.006.patch > Harden edit log reading code against out of memory er

[jira] [Commented] (HDFS-8965) Harden edit log reading code against out of memory errors

[ https://issues.apache.org/jira/browse/HDFS-8965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14723839#comment-14723839 ] Colin Patrick McCabe commented on HDFS-8965: bq. Currently jenkins report will

[jira] [Commented] (HDFS-8965) Harden edit log reading code against out of memory errors

[ https://issues.apache.org/jira/browse/HDFS-8965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14723831#comment-14723831 ] Colin Patrick McCabe commented on HDFS-8965: bq. I think it will be easy to und

[jira] [Commented] (HDFS-8965) Harden edit log reading code against out of memory errors

[ https://issues.apache.org/jira/browse/HDFS-8965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720699#comment-14720699 ] Colin Patrick McCabe commented on HDFS-8965: bq. Do we also already have tests

[jira] [Updated] (HDFS-8965) Harden edit log reading code against out of memory errors

[ https://issues.apache.org/jira/browse/HDFS-8965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8965: --- Attachment: HDFS-8965.005.patch > Harden edit log reading code against out of memory er

<    1   2   3   4   5   6   7   8   9   10   >