Re: [Lustre-discuss] noatime or atime_diff for Lustre 1.8.7?

2012-12-07 Thread Mark Day
> 2) Make sure caching is enabled on the oss. How do you check/enable for this? Is it not enabled by default? Cheers, Mark - Original Message - From: "Mohr Jr, Richard Frank (Rick Mohr)" To: "Grigory Shamov" Cc: lustre-discuss@lists.lustre.org Sent: Saturday, 8 December, 2012 5:

Re: [Lustre-discuss] noatime or atime_diff for Lustre 1.8.7?

2012-12-07 Thread Mohr Jr, Richard Frank (Rick Mohr)
On Dec 6, 2012, at 2:58 PM, Grigory Shamov wrote: > So, on one of our OSS servers the load is now 160. According to collectl, > only one OST does most of the job. (We dont do striping on this FS; unless > users to it manually on their subdirectories). This sounds similar to situations we see ev

Re: [Lustre-discuss] Applications of Lustre - streaming?

2012-12-07 Thread Jeff Johnson
On 12/7/12 9:34 AM, Dilger, Andreas wrote: > I've been using Lustre for years with my home MythTV (Linux PVR) setup. Nerd. :) -- -- Jeff Johnson Co-Founder Aeon Computing jeff.john...@aeoncomputing.com www.aeoncomputing.com t: 858-412-3810 x101 f: 858-412-3845 m: 61

Re: [Lustre-discuss] Applications of Lustre - streaming?

2012-12-07 Thread Jason Brooks
It is the question of how to handle redundancy that stops me from immediately testing this idea of mine. Well, that and time, werewithal, etc... Hadoop is great because it uses the speed and latency of local disks to work with data, and does not require systems be homogeneous. With the data repl

Re: [Lustre-discuss] Applications of Lustre - streaming?

2012-12-07 Thread Jason Brooks
I have used fuse for other filesystems: its great if all you need is access to the data, but the performance is HORRIBLE. --jason From: Jon Yeargers mailto:yearg...@ohsu.edu>> Date: Friday, December 7, 2012 9:42 AM To: Jason Brooks mailto:brook...@ohsu.edu>>, "lustre-discuss@lists.lustre.org

Re: [Lustre-discuss] Applications of Lustre - streaming?

2012-12-07 Thread Jon Yeargers
If it weren’t for the positive aspects of HDFS I wouldn’t really be considering HBase (over Cassandra). Any notion of the merits of Lustre’s kernel-based mounts vs a FUSE-based mount (HDFS)? Whichever filesystem I go with I will need to store ‘flat files’ in. From: Jason Brooks Sent: Friday, De

Re: [Lustre-discuss] Applications of Lustre - streaming?

2012-12-07 Thread Jason Brooks
Hello, The question of hdfs storage via lustre has been in the foreground of my thinking. the hadoop hdfs processes are not aware of block devices: they only know of a filesystem mount point to begin storing data in hdfs. THUS… If we provide a filesystem interface (say a lustre mount point) w

Re: [Lustre-discuss] Applications of Lustre - streaming?

2012-12-07 Thread Jon Yeargers
The redundancy of HDFS is very appealing. I've been weighing the merits of this vs a RAID-6 / server on Lustre. HDFS recommends avoiding RAID for the very reason that the data is (typically) saved in several locations. -Original Message- From: Dilger, Andreas [mailto:andreas.dil...@inte

Re: [Lustre-discuss] Applications of Lustre - streaming?

2012-12-07 Thread Dilger, Andreas
On 2012-12-07, at 10:26, Jon Yeargers mailto:yearg...@ohsu.edu>> wrote: Can Lustre be used to store data like streaming audio / video? I’ve been scolded about considering it for DB storage but I’m looking at the relative merits of Lustre vs HDFS. I've been using Lustre for years with my home My

Re: [Lustre-discuss] Applications of Lustre - streaming?

2012-12-07 Thread Colin Faber
Hi, On 12/07/2012 10:26 AM, Jon Yeargers wrote: > > Can Lustre be used to store data like streaming audio / video? > Yes > I’ve been scolded about considering it for DB storage but I’m looking > at the relative merits of Lustre vs HDFS. > db reads/writes tends to lead to small I/O which lustre d

[Lustre-discuss] Applications of Lustre - streaming?

2012-12-07 Thread Jon Yeargers
Can Lustre be used to store data like streaming audio / video? I’ve been scolded about considering it for DB storage but I’m looking at the relative merits of Lustre vs HDFS. I’m moving to a clustered DB setup and wondering about Cassandra / Lustre vs Hadoop (IE HBase / HDFS). One offers flexib

Re: [Lustre-discuss] New Test Framework Development - Requirements Capture

2012-12-07 Thread Gearing, Chris
Just a reminder that we have a meeting on Tuesday to discuss the new test framework development and that the wiki page on the opensfs site is ready for capturing everybody's thoughts. The plan for the next meeting to be to discuss the requirements that have been captured. Thanks Chris Call Info