[
https://issues.apache.org/jira/browse/HADOOP-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638927#action_12638927
]
Pete Wyckoff commented on HADOOP-4397:
--------------------------------------
thanks Brian.
Let's get this patch into 0.18.2 and something like this into 0.19 (if possible
as this should be a blocker).
And then, right, work on something better.
One question though, it doesn't look like you protected all writes to the dfs
context in other operations. Also, do you have a test case for this? I know
it's nearly impossible :) .
We should probably work on 0.19 patch with the highest priority since it may be
released soon (or maybe has ?).
pete
> fuse-dfs causes corruptions on multi-threaded access
> ----------------------------------------------------
>
> Key: HADOOP-4397
> URL: https://issues.apache.org/jira/browse/HADOOP-4397
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fuse-dfs
> Affects Versions: 0.18.1
> Reporter: Brian Bockelman
> Fix For: 0.18.2
>
> Attachments: hadoop-4397.patch
>
>
> If multiple threads in the same process perform file system reads, then
> fuse-dfs causes various problems due to the per-context buffer. I've seen
> this reflected in segmentation violations and corruptions.
> I'll attach a proposed patch which takes the "easy way" out - I surround all
> calls to dfs_read with a mutex. You will obviously get performance
> degradations through thrashing if the threads are reading different parts of
> the file (but for our application, the multi-threaded reads are very, very
> infrequent.
> If we want to have fuse-dfs writes/reads in 0.19 or 0.20, we'll probably need
> to do the same thing with writes.
> This patch could be easily integrated as stands, or a more elaborate approach
> could be taken - per-thread buffers maybe?
> Thanks as always for looking into this,
> Brian
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
