fuse-dfs causes corruptions on multi-threaded access
----------------------------------------------------
Key: HADOOP-4397
URL: https://issues.apache.org/jira/browse/HADOOP-4397
Project: Hadoop Core
Issue Type: Bug
Components: contrib/fuse-dfs
Affects Versions: 0.18.1
Reporter: Brian Bockelman
Fix For: 0.18.2
If multiple threads in the same process perform file system reads, then
fuse-dfs causes various problems due to the per-context buffer. I've seen this
reflected in segmentation violations and corruptions.
I'll attach a proposed patch which takes the "easy way" out - I surround all
calls to dfs_read with a mutex. You will obviously get performance
degradations through thrashing if the threads are reading different parts of
the file (but for our application, the multi-threaded reads are very, very
infrequent.
If we want to have fuse-dfs writes/reads in 0.19 or 0.20, we'll probably need
to do the same thing with writes.
This patch could be easily integrated as stands, or a more elaborate approach
could be taken - per-thread buffers maybe?
Thanks as always for looking into this,
Brian
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
