[jira] [Updated] (HDFS-8272) Erasure Coding: simplify the retry logic in DFSStripedInputStream (stateful read)
[ https://issues.apache.org/jira/browse/HDFS-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8272: Summary: Erasure Coding: simplify the retry logic in DFSStripedInputStream (stateful read) (was: Erasure Coding: simplify the retry logic in DFSStripedInputStream) Erasure Coding: simplify the retry logic in DFSStripedInputStream (stateful read) - Key: HDFS-8272 URL: https://issues.apache.org/jira/browse/HDFS-8272 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Jing Zhao Assignee: Jing Zhao Attachments: h8272-HDFS-7285.000.patch, h8272-HDFS-7285.001.patch Currently in DFSStripedInputStream the retry logic is still the same with DFSInputStream. More specifically, every failed read will try to search for another source node. And an exception is thrown when no new source node can be identified. This logic is not appropriate for EC inputstream and can be simplified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8272) Erasure Coding: simplify the retry logic in DFSStripedInputStream (stateful read)
[
https://issues.apache.org/jira/browse/HDFS-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jing Zhao updated HDFS-8272:
Attachment: HDFS-8272.002.patch
Thanks for the review, Zhe and Yi! I just merged HDFS-8280 into HDFS-7285
branch. Upload the patch excluding the changes in DFSInputStream. Also fix the
bug pointed out by Yi.
About the {{seekToBlockSource}}, I think it may be better to remove it by now:
# With decoding functionality we do not need to spend more time trying our luck
on the same DN.
# Currently calling {{seekToBlockSource}} will cause all the current block
readers to be closed (since {{blockSeekTo}} will call
{{closeCurrentBlockReaders}}). To fix this we need to add extra complexity so
as to make sure only one block reader is retried.
Erasure Coding: simplify the retry logic in DFSStripedInputStream (stateful
read)
-
Key: HDFS-8272
URL: https://issues.apache.org/jira/browse/HDFS-8272
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Jing Zhao
Assignee: Jing Zhao
Attachments: HDFS-8272.002.patch, h8272-HDFS-7285.000.patch,
h8272-HDFS-7285.001.patch
Currently in DFSStripedInputStream the retry logic is still the same with
DFSInputStream. More specifically, every failed read will try to search for
another source node. And an exception is thrown when no new source node can
be identified. This logic is not appropriate for EC inputstream and can be
simplified.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8272) Erasure Coding: simplify the retry logic in DFSStripedInputStream
[
https://issues.apache.org/jira/browse/HDFS-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jing Zhao updated HDFS-8272:
Attachment: h8272-HDFS-7285.001.patch
Thanks again for the review, Zhe! Update the patch to address your comments
(including DFSInputStream changes). The main change is to only refetch the
key/token once for the group.
About the encryption key retry logic, I think it is handled while creating the
block reader. More specifically, while creating the TCP peer in
{{BlockReaderFactory#getRemoteBlockReaderFromTcp}}, the sasl protocol is
triggered during which the encryptionKey can be refetched.
Erasure Coding: simplify the retry logic in DFSStripedInputStream
-
Key: HDFS-8272
URL: https://issues.apache.org/jira/browse/HDFS-8272
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Jing Zhao
Assignee: Jing Zhao
Attachments: h8272-HDFS-7285.000.patch, h8272-HDFS-7285.001.patch
Currently in DFSStripedInputStream the retry logic is still the same with
DFSInputStream. More specifically, every failed read will try to search for
another source node. And an exception is thrown when no new source node can
be identified. This logic is not appropriate for EC inputstream and can be
simplified.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
