Yes, I am referring to HDFS taking multiple mounts points and automatically 
round-robin block allocation across it.
A single file block will only exist on a single disk, but the extra speed you 
can get with raid-0 within a block can't be used effectively by almost any 
mapper or reducer anyway.  Perhaps an identity mapper can read faster than a 
single disk - but certainly not if the content is compressed.  \

RAID-0 may be more useful for local temp space.

In effect, you can say that HDFS data nodes already do RAID-0, but with a very 
large block size, and where failure of a disk reduces the redundancy minimally 
and temporarily.

For reference, today's Intel / AMD CPUs can decompress a gzip stream at less 
than 30MB/sec usually  (50MB to 100MB of uncompressed data output a sec).


On 3/14/09 1:53 AM, "Vadim Zaliva" <[email protected]> wrote:

Scott,

Thanks for interesting information. By JBOD, I assume you mean just listing
multiple partition mount points in hadoop config?

Vadim

On Fri, Mar 13, 2009 at 12:48, Scott Carey <[email protected]> wrote:
> On 3/13/09 11:56 AM, "Allen Wittenauer" <[email protected]> wrote:
>
> On 3/13/09 11:25 AM, "Vadim Zaliva" <[email protected]> wrote:
>
>>>    When you stripe you automatically make every disk in the system have the
>>> same speed as the slowest disk.  In our experiences, systems are more likely
>>> to have a 'slow' disk than a dead one.... and detecting that is really
>>> really hard.  In a distributed system, that multiplier effect can have
>>> significant consequences on the whole grids performance.
>>
>> All disk are the same, so there is no speed difference.
>
>    There will be when they start to fail. :)
>
>
>
> This has been discussed before:
> http://www.nabble.com/RAID-vs.-JBOD-td21404366.html
>
> JBOD is going to be better, the only benefit of RAID-0 is slightly easier 
> management in hadoop config, but harder to manage at the OS level.
> When a single JBOD drive dies, you only lose that set of data.  The datanode 
> goes down but a restart brings back up the parts that still exist.  Then you 
> can leave it be while the replacement is procured... With RAID-0 the whole 
> node is down until you get the new drive and recreate the RAID.
>
> With JBOD, don't forget to set the linux readahead for the drives to a decent 
> level  (you'll gain up to 25% more sequential read throughput depending on 
> your kernel version).  (blockdev -setra 8192 /dev/<device>).  I also see good 
> gains by using xfs instead of ext3.  For a big shocker check out the 
> difference in time to delete a bunch of large files with ext3 (long time) 
> versus xfs (almost instant).
>
> For the newer drives, they can do about 120MB/sec at the front of the drive 
> when tuned (xfs, readahead >4096) and the back of the drive is 60MB/sec.  If 
> you are going to not use 100% of the drive for HDFS, use this knowledge and 
> place the partitions appropriately.  The last 20% or so of the drive is a lot 
> slower than the front 60%.  Here is a typical sequential transfer rate chart 
> for a SATA drive as a function of LBA:
> http://www.tomshardware.com/reviews/Seagate-Barracuda-1.5-TB,2032-5.html
> (graphs aare about 3/4 of the way down the page before the comments).
>

Reply via email to