[
https://issues.apache.org/jira/browse/HADOOP-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pete Wyckoff updated HADOOP-4397:
---------------------------------
Attachment: HADOOP-4397.2.txt
I suggest we use this patch which fixes both bugs and is the version i intend
to submit to trunk - it needs to be compiled with the extra flag
-Dlibhdfs.noperms=1 so that 0.19 new permission functions in libhdfs are not
used. I actually use this version of the code with 0.17 in production.
of course the unit tests fail for chown and chmod (since they are not
implemented) with this but all else passes.
and i hope it fixes all the concurrency problems including system calls that
return static structs and so need to be protected.
> fuse-dfs causes corruptions on multi-threaded access
> ----------------------------------------------------
>
> Key: HADOOP-4397
> URL: https://issues.apache.org/jira/browse/HADOOP-4397
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fuse-dfs
> Affects Versions: 0.18.1
> Reporter: Brian Bockelman
> Fix For: 0.18.2
>
> Attachments: HADOOP-4397.2.txt, hadoop-4397.patch
>
>
> If multiple threads in the same process perform file system reads, then
> fuse-dfs causes various problems due to the per-context buffer. I've seen
> this reflected in segmentation violations and corruptions.
> I'll attach a proposed patch which takes the "easy way" out - I surround all
> calls to dfs_read with a mutex. You will obviously get performance
> degradations through thrashing if the threads are reading different parts of
> the file (but for our application, the multi-threaded reads are very, very
> infrequent.
> If we want to have fuse-dfs writes/reads in 0.19 or 0.20, we'll probably need
> to do the same thing with writes.
> This patch could be easily integrated as stands, or a more elaborate approach
> could be taken - per-thread buffers maybe?
> Thanks as always for looking into this,
> Brian
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
