On 9/9/14, 5:06 AM, Tomas Meszaros wrote:
Has anyone tried to run Tez with different filesystem than HDFS (like
glusterfs)?
I'm wondering if someone successfully tried it.
Tez should support anything that comes with a hadoop.fs.FileSystem
interface.
I did play with Ceph for a bit, but only on my dev box.
But since I was on Hadoop-2.4, I mounted an RBD and used it as a
RawLocalFileSystem.
That is not a good idea.
The part I feel is suspect in this matter is the dynamic split grouping
- which relies on locality scheduling.
HDFS does provide locality information through its abstraction, but I'm
not sure how we can do that for something that gets locally mounted
instead of being accessed via LocatedFileStatus.
If you have questions or need help, don't hesitate to ask this list.
Cheers,
Gopal
