[
https://issues.apache.org/jira/browse/HDFS-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Zhang updated HDFS-9098:
----------------------------
Attachment: HDFS-9098.00.patch
Thanks Jing for the helpful review! Updating the patch to address the comment
on linking fault injection and sync points.
The basic challenge here is to handle non-linear event flow in a thread (e.g.
with if statements). The approach in the patch is to have 2 types of sync
points:
* {{syncBarrier}}: blocks until the prior event happens
* {{syncCondition}}: returns false until the prior event happens
The biggest remaining work is to add more details in each sync event (either
barrier or condition). For example, a certain {{WRITE_CHUNK}} barrier should
only apply for a specified file offset.
This is still an experimental patch, so more comments / questions will be very
helpful.
> Erasure coding: emulate race conditions among striped streamers in write
> pipeline
> ---------------------------------------------------------------------------------
>
> Key: HDFS-9098
> URL: https://issues.apache.org/jira/browse/HDFS-9098
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: erasure-coding
> Affects Versions: 3.0.0
> Reporter: Zhe Zhang
> Assignee: Zhe Zhang
> Attachments: HDFS-9098.00.patch, HDFS-9098.wip.patch
>
>
> Apparently the interleaving of events among {{StripedDataStreamer}}'s is very
> tricky to handle. [~walter.k.su] and [~jingzhao] have discussed several race
> conditions under HDFS-9040.
> Let's use FaultInjector to emulate different combinations of interleaved
> events.
> In particular, we should consider inject delays in the following places:
> # {{Streamer#endBlock}}
> # {{Streamer#locateFollowingBlock}}
> # {{Streamer#updateBlockForPipeline}}
> # {{Streamer#updatePipeline}}
> # {{OutputStream#writeChunk}}
> # {{OutputStream#close}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
