I have used fuse for other filesystems: its great if all you need is access to the data, but the performance is HORRIBLE.
--jason From: Jon Yeargers <yearg...@ohsu.edu<mailto:yearg...@ohsu.edu>> Date: Friday, December 7, 2012 9:42 AM To: Jason Brooks <brook...@ohsu.edu<mailto:brook...@ohsu.edu>>, "lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>" <lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>> Subject: RE: [Lustre-discuss] Applications of Lustre - streaming? If it weren’t for the positive aspects of HDFS I wouldn’t really be considering HBase (over Cassandra). Any notion of the merits of Lustre’s kernel-based mounts vs a FUSE-based mount (HDFS)? Whichever filesystem I go with I will need to store ‘flat files’ in. From: Jason Brooks Sent: Friday, December 07, 2012 9:38 AM To: Jon Yeargers; lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> Subject: Re: [Lustre-discuss] Applications of Lustre - streaming? Hello, The question of hdfs storage via lustre has been in the foreground of my thinking. the hadoop hdfs processes are not aware of block devices: they only know of a filesystem mount point to begin storing data in hdfs. THUS… If we provide a filesystem interface (say a lustre mount point) whose latencies and throughput approach that of local disk storage (say, via infiniband), could we not have the various hadoop nodes store their data in the lustre filesystem? would hadoop even care? I realize that this may not be a good place to bring it up. But there you go… One of these days, (with all of my ample spare time), I will benchmark it. and report of course… --jason From: Jon Yeargers <yearg...@ohsu.edu<mailto:yearg...@ohsu.edu>> Date: Friday, December 7, 2012 9:26 AM To: "lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>" <lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>> Subject: [Lustre-discuss] Applications of Lustre - streaming? Can Lustre be used to store data like streaming audio / video? I’ve been scolded about considering it for DB storage but I’m looking at the relative merits of Lustre vs HDFS. I’m moving to a clustered DB setup and wondering about Cassandra / Lustre vs Hadoop (IE HBase / HDFS). One offers flexibility in terms of mixing hardware components while the other is a ‘one stop shop’. Not trying to elicit a religious war – and yes, I’ve been reading as much as I can find about this. Just hoping for the opinion(s) of this side of the table.
_______________________________________________ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
