[
https://issues.apache.org/jira/browse/HDFS-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16152037#comment-16152037
]
He Tianyi commented on HDFS-10467:
--
[~elgoiri], thanks for asking.
our federated cluster
[
https://issues.apache.org/jira/browse/HDFS-10490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15326364#comment-15326364
]
He Tianyi commented on HDFS-10490:
--
I believe the whitespace and checkstyle issue is not
[
https://issues.apache.org/jira/browse/HDFS-10490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-10490:
-
Attachment: HDFS-10490.0001.patch
Added test.
> Client may never recovery replica after a timeout during
[
https://issues.apache.org/jira/browse/HDFS-10331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15326218#comment-15326218
]
He Tianyi commented on HDFS-10331:
--
Is there any code path (e.g. off heap buffer) still u
[
https://issues.apache.org/jira/browse/HDFS-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317636#comment-15317636
]
He Tianyi commented on HDFS-10467:
--
+1. This also reduces latency for first request from
[
https://issues.apache.org/jira/browse/HDFS-10490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317587#comment-15317587
]
He Tianyi commented on HDFS-10490:
--
Thanks for pointing that out, [~kihwal].
I think the
[
https://issues.apache.org/jira/browse/HDFS-10490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-10490:
-
Status: Patch Available (was: Open)
> Client may never recovery replica after a timeout during sending pa
[
https://issues.apache.org/jira/browse/HDFS-10490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-10490:
-
Attachment: HDFS-10490.patch
> Client may never recovery replica after a timeout during sending packet
> -
He Tianyi created HDFS-10490:
Summary: Client may never recovery replica after a timeout during
sending packet
Key: HDFS-10490
URL: https://issues.apache.org/jira/browse/HDFS-10490
Project: Hadoop HDFS
[
https://issues.apache.org/jira/browse/HDFS-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15305716#comment-15305716
]
He Tianyi commented on HDFS-10467:
--
Thanks for sharing.
I've implemented similar approac
[
https://issues.apache.org/jira/browse/HDFS-10331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-10331:
-
Description:
In java8, performance of intrinsic CRC32 has been dramatically improved.
See: https://bugs.op
[
https://issues.apache.org/jira/browse/HDFS-10331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15259182#comment-15259182
]
He Tianyi commented on HDFS-10331:
--
Yes, okay. Now I see that for array-backed buffers, n
[
https://issues.apache.org/jira/browse/HDFS-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15259173#comment-15259173
]
He Tianyi commented on HDFS-10326:
--
[~cmccabe] [~mingma]
https://www.kernel.org/doc/ols/2
[
https://issues.apache.org/jira/browse/HDFS-10331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15258218#comment-15258218
]
He Tianyi commented on HDFS-10331:
--
Forgot to add, on same platform.
The new intrinsic CR
He Tianyi created HDFS-10331:
Summary: Use java.util.zip.CRC32 for checksum in java8 or above
Key: HDFS-10331
URL: https://issues.apache.org/jira/browse/HDFS-10331
Project: Hadoop HDFS
Issue Type
[
https://issues.apache.org/jira/browse/HDFS-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15257422#comment-15257422
]
He Tianyi commented on HDFS-10326:
--
Some system may not support auto tuning, defaulting t
[
https://issues.apache.org/jira/browse/HDFS-9412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15244578#comment-15244578
]
He Tianyi commented on HDFS-9412:
-
Hi, [~walter.k.su]. Would you commit this patch when ava
He Tianyi created HDFS-10290:
Summary: Move getBlocks calls to DataNode in Balancer
Key: HDFS-10290
URL: https://issues.apache.org/jira/browse/HDFS-10290
Project: Hadoop HDFS
Issue Type: New Feat
[
https://issues.apache.org/jira/browse/HDFS-9412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-9412:
Attachment: HDFS-9412.0002.patch
Fix codestyle, whitespace and unit test.
> getBlocks occupies FSLock and ta
[
https://issues.apache.org/jira/browse/HDFS-9412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-9412:
Attachment: HDFS-9412.0001.patch
Rebase against current trunk.
> getBlocks occupies FSLock and takes too lon
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15124825#comment-15124825
]
He Tianyi commented on HDFS-8829:
-
Sorry for late response.
There is one corner cases: Old
[
https://issues.apache.org/jira/browse/HDFS-9412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15124822#comment-15124822
]
He Tianyi commented on HDFS-9412:
-
Added ~200 nodes to HDFS cluster recently. As balancing
He Tianyi created HDFS-9725:
---
Summary: Make capacity of centralized cache dynamic
Key: HDFS-9725
URL: https://issues.apache.org/jira/browse/HDFS-9725
Project: Hadoop HDFS
Issue Type: Wish
[
https://issues.apache.org/jira/browse/HDFS-9665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi resolved HDFS-9665.
-
Resolution: Duplicate
> Cilents are easily affected by standby namenode
> -
[
https://issues.apache.org/jira/browse/HDFS-9665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107896#comment-15107896
]
He Tianyi commented on HDFS-9665:
-
Thanks. That patch worked well.
Closing issue.
> Cilen
[
https://issues.apache.org/jira/browse/HDFS-9665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107854#comment-15107854
]
He Tianyi commented on HDFS-9665:
-
Also, I observed that {{doTailEdits}} may acquire write
He Tianyi created HDFS-9665:
---
Summary: Cilents are easily affected by standby namenode
Key: HDFS-9665
URL: https://issues.apache.org/jira/browse/HDFS-9665
Project: Hadoop HDFS
Issue Type: Wish
[
https://issues.apache.org/jira/browse/HDFS-9412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-9412:
Status: Patch Available (was: Open)
> getBlocks occupies FSLock and takes too long to complete
> ---
[
https://issues.apache.org/jira/browse/HDFS-9412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-9412:
Attachment: HDFS-9412..patch
A patch that skip small blocks in {{getBlocks}}. These are unused anyway.
Th
[
https://issues.apache.org/jira/browse/HDFS-9412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046417#comment-15046417
]
He Tianyi commented on HDFS-9412:
-
[~andrew.wang] Perhaps switching to unfair RWLock may ca
[
https://issues.apache.org/jira/browse/HDFS-9412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15000269#comment-15000269
]
He Tianyi commented on HDFS-9412:
-
A simpler approach would be ignore small blocks during i
[
https://issues.apache.org/jira/browse/HDFS-9412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15000242#comment-15000242
]
He Tianyi commented on HDFS-9412:
-
Sorry the misuse of word "starvation". It's not starvati
[
https://issues.apache.org/jira/browse/HDFS-9412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-9412:
Description:
{{getBlocks}} in {{NameNodeRpcServer}} acquires a read lock then may take a
long time to comple
He Tianyi created HDFS-9412:
---
Summary: getBlocks occupies FSLock and takes too long to complete
Key: HDFS-9412
URL: https://issues.apache.org/jira/browse/HDFS-9412
Project: Hadoop HDFS
Issue Type:
[
https://issues.apache.org/jira/browse/HDFS-9071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-9071:
Attachment: HDFS-9071.0004.patch
Fix jenkins
> chooseTargets in ReplicationWork may pass incomplete srcPath
[
https://issues.apache.org/jira/browse/HDFS-9071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-9071:
Attachment: HDFS-9071.0003.patch
Rebase.
> chooseTargets in ReplicationWork may pass incomplete srcPath
> --
[
https://issues.apache.org/jira/browse/HDFS-9149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14942031#comment-14942031
]
He Tianyi commented on HDFS-9149:
-
Perhaps we can either go explicit approach or implicit o
[
https://issues.apache.org/jira/browse/HDFS-9071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-9071:
Attachment: HDFS-9071.0001.patch
Kick jenkins again.
> chooseTargets in ReplicationWork may pass incomplete
[
https://issues.apache.org/jira/browse/HDFS-9081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14942008#comment-14942008
]
He Tianyi commented on HDFS-9081:
-
I think code below in {{waitForAckedSeqno}} is already a
[
https://issues.apache.org/jira/browse/HDFS-9149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941264#comment-14941264
]
He Tianyi commented on HDFS-9149:
-
Thanks, [~hexiaoqiao].
The simpler idea sounds good! Bu
[
https://issues.apache.org/jira/browse/HDFS-9149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941100#comment-14941100
]
He Tianyi commented on HDFS-9149:
-
I think that's a good point [~hexiaoqiao].
One simple i
[
https://issues.apache.org/jira/browse/HDFS-9149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi reassigned HDFS-9149:
---
Assignee: He Tianyi
> Consider multi datacenter when sortByDistance
>
[
https://issues.apache.org/jira/browse/HDFS-9149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14909740#comment-14909740
]
He Tianyi commented on HDFS-9149:
-
The default {{NetworkTopology}} is not dc aware. Perhaps
[
https://issues.apache.org/jira/browse/HDFS-9071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14905613#comment-14905613
]
He Tianyi commented on HDFS-9071:
-
Thanks for your time, [~andreina].
I think there may be
[
https://issues.apache.org/jira/browse/HDFS-9071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-9071:
Status: Patch Available (was: Open)
> chooseTargets in ReplicationWork may pass incomplete srcPath
> ---
[
https://issues.apache.org/jira/browse/HDFS-9071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-9071:
Attachment: HDFS-9071.0001.patch
Adds precondition to make sure {{chooseTargets}} receives either complete pa
[
https://issues.apache.org/jira/browse/HDFS-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14900148#comment-14900148
]
He Tianyi commented on HDFS-9090:
-
The combination theory makes sense. Thanks guys.
Since
[
https://issues.apache.org/jira/browse/HDFS-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14791457#comment-14791457
]
He Tianyi commented on HDFS-9090:
-
Not quite sure but I do think perhaps this is perpendicu
[
https://issues.apache.org/jira/browse/HDFS-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14769041#comment-14769041
]
He Tianyi commented on HDFS-9090:
-
Alternatively, from a different perspective, perhaps we
[
https://issues.apache.org/jira/browse/HDFS-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14768973#comment-14768973
]
He Tianyi commented on HDFS-9090:
-
Thanks, [~ste...@apache.org].
My case may be a little r
[
https://issues.apache.org/jira/browse/HDFS-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-9090:
Description:
(I am not sure whether this should be reported as BUG, feel free to modify this)
Current block
He Tianyi created HDFS-9090:
---
Summary: Write hot data on few nodes may cause performance issue
Key: HDFS-9090
URL: https://issues.apache.org/jira/browse/HDFS-9090
Project: Hadoop HDFS
Issue Type: B
[
https://issues.apache.org/jira/browse/HDFS-9075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746675#comment-14746675
]
He Tianyi commented on HDFS-9075:
-
Thanks for point that out, [~cnauroth].
Prior discusses
[
https://issues.apache.org/jira/browse/HDFS-9081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-9081:
Summary: False-positive ACK slow log in DFSClient (was: False-positive of
ACK slow log in DFSClient)
> Fals
He Tianyi created HDFS-9081:
---
Summary: False-positive of ACK slow log in DFSClient
Key: HDFS-9081
URL: https://issues.apache.org/jira/browse/HDFS-9081
Project: Hadoop HDFS
Issue Type: Bug
Affec
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-8829:
Attachment: HDFS-8829.0006.patch
Rebased again current trunk.
> DataNode sets SO_RCVBUF explicitly is disabl
[
https://issues.apache.org/jira/browse/HDFS-9075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-9075:
Description:
It is common scenario of deploying multiple datacenter for scaling and disaster
tolerant.
In t
He Tianyi created HDFS-9075:
---
Summary: Multiple datacenter replication inside one HDFS cluster
Key: HDFS-9075
URL: https://issues.apache.org/jira/browse/HDFS-9075
Project: Hadoop HDFS
Issue Type: N
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14742846#comment-14742846
]
He Tianyi commented on HDFS-8829:
-
* "I've" -> "I'll"
> DataNode sets SO_RCVBUF explicitly
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14742844#comment-14742844
]
He Tianyi commented on HDFS-8829:
-
Hi, thanks for the comments.
I've update the patch agai
He Tianyi created HDFS-9071:
---
Summary: chooseTargets in ReplicationWork may pass incomplete
srcPath
Key: HDFS-9071
URL: https://issues.apache.org/jira/browse/HDFS-9071
Project: Hadoop HDFS
Issue T
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-8829:
Attachment: HDFS-8829.0005.patch
Fix codestyle.
> DataNode sets SO_RCVBUF explicitly is disabling tcp auto-t
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-8829:
Attachment: HDFS-8829.0004.patch
Hi, sorry for late response.
Agree with you, [~cmccabe].
I've updated patch
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14716471#comment-14716471
]
He Tianyi commented on HDFS-8829:
-
Is there anything wrong with CI?
I've run {{test-patch.s
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-8829:
Attachment: HDFS-8829.0003.patch
Yes, right.
Thanks for comments.
I've updated the patch.
> DataNode sets S
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14710434#comment-14710434
]
He Tianyi commented on HDFS-8829:
-
Hi [~cmccabe],
About the second comment:
Actually {{DEF
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708270#comment-14708270
]
He Tianyi commented on HDFS-8829:
-
I am confused with this checkstyle warning:
{{./hadoop-h
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-8829:
Attachment: HDFS-8829.0002.patch
Added configuration to hdfs-default.xml
> DataNode sets SO_RCVBUF explicitl
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-8829:
Status: Patch Available (was: Open)
Added *dfs.data.socket.size*, with default value 128KB.
When set to 0 or
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-8829:
Attachment: HDFS-8829.0001.patch
Added *dfs.data.socket.size*, with default value 128KB.
When set to 0 or neg
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi reassigned HDFS-8829:
---
Assignee: He Tianyi (was: kanaka kumar avvaru)
> DataNode sets SO_RCVBUF explicitly is disabling tcp
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704266#comment-14704266
]
He Tianyi commented on HDFS-8829:
-
Completely agree with you, [~cmccabe].
> DataNode sets
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702365#comment-14702365
]
He Tianyi commented on HDFS-8829:
-
I think auto-tuning does the magic when network latency
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14650141#comment-14650141
]
He Tianyi commented on HDFS-8829:
-
Hi kanaka kumar avvaru,
I've applied the improvement to
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14645305#comment-14645305
]
He Tianyi commented on HDFS-8829:
-
This affects pipeline throughput, and I've observed a 30
[
https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Tianyi updated HDFS-8829:
Description:
{code:java}
private void initDataXceiver(Configuration conf) throws IOException {
// find
He Tianyi created HDFS-8829:
---
Summary: DataNode sets SO_RCVBUF explicitly is disabling tcp
auto-tuning
Key: HDFS-8829
URL: https://issues.apache.org/jira/browse/HDFS-8829
Project: Hadoop HDFS
Issu
77 matches
Mail list logo
