On 18 October 2012 16:46, Zhani Pellumbi zhp9...@nyp.org wrote:
Yes, Isilon NAS runs HDFS natively- thus your nodes become compute
nodes, running only task tracker processes.
I read the NetApp paper, and this is fundamentally different architecture
though.
There are some obvious benefits ,
I haven't played with a NetApp box, but the way it has been explained to me is
that your SAN appears as if its direct attached storage.
Its possible, based on drives and other hardware, plus it looks like they are
focusing on read times only.
I'd contact a NetApp rep for a better answer.
/2012 05:33 AM
Subject:Re: HDFS using SAN
On 10/18/2012 02:21 AM, Pamecha, Abhishek wrote:
Tom
Do you mean you are using GPFS instead of HDFS? Also, if you can share,
are you deploying it as DAS set up or a SAN?
Thanks,
Abhishek
Though I don't think I'd buy a SAN for a new Hadoop
I wonder if large NAS equipment manufacturers have ever considered modifying
their firmware to directly talk the DFS protocol that hadoop uses. This way
your compute nodes could be 'pure' compute nodes with only tasktracker
processes.
Might be a way to extend their market a bit. Not sure it
Pireddu pire...@crs4.itmailto:pire...@crs4.it
To: user@hadoop.apache.orgmailto:user@hadoop.apache.org,
Date: 10/18/2012 05:33 AM
Subject: Re: HDFS using SAN
On 10/18/2012 02:21 AM, Pamecha, Abhishek wrote:
Tom
Do you mean you are using GPFS instead of HDFS
6:49 AM
To: user@hadoop.apache.org
Subject: Re: HDFS using SAN
Hi,
In the NetApp whitepaper on SAN solution (link given by Kevin) it makes
following statement. Can someone please elaborate (or give a link that
explains) how 12-disk in SAN can give 2000 IOPS while if used as JBOD would
give 600
.
** **
Thanks,
Abhishek
** **
*From:* lohit [mailto:lohit.vijayar...@gmail.com]
*Sent:* Tuesday, October 16, 2012 3:26 PM
*To:* user@hadoop.apache.org
*Subject:* Re: HDFS using SAN
** **
Adding to this. Locality is very important for MapReduce applications. One
might not see
09:25 AM AST
To: user@hadoop.apache.org
Subject: Re: HDFS using SAN
You may want to take a look at the Netapp White Paper on this. They have a
SAN solution as their Hadoop offering.
http://www.netapp.com/templates/mediaView?m=tr-3969.pdfcc=uswid=130618138mid=56872393
On Tue, Oct 16, 2012 at 7
?
3 way Replication and RAID suggestions are great.
Thanks,
Abhishek
From: lohit [mailto:lohit.vijayar...@gmail.com]
Sent: Tuesday, October 16, 2012 3:26 PM
To: user@hadoop.apache.org
Subject: Re: HDFS using SAN
Adding to this. Locality is very important for MapReduce
Tom
Do you mean you are using GPFS instead of HDFS? Also, if you can share, are you
deploying it as DAS set up or a SAN?
Thanks,
Abhishek
From: Tom Deutsch [mailto:tdeut...@us.ibm.com]
Sent: Wednesday, October 17, 2012 6:31 AM
To: user
Subject: Re: HDFS using SAN
And of source IBM has
fault-tolerance when its controller(s) fail.
Thanks,
Abhishek
From: Mohamed Riadh Trad [mailto:mohamed.t...@inria.fr]
Sent: Wednesday, October 17, 2012 6:37 AM
To: user@hadoop.apache.org
Subject: Re: HDFS using SAN
Sauvegarde tes données!
Le 17 oct. 2012 à 15:25, Kevin O'dell a écrit :
You may
Hi
I have read scattered documentation across the net which mostly say HDFS
doesn't go well with SAN being used to store data. While some say, it is an
emerging trend. I would love to know if there have been any tests performed
which hint on what aspects does a direct storage excels/falls
is a bit bleeding-edge.
Jeff
From: Pamecha, Abhishek [mailto:apame...@x.com]
Sent: Tuesday, October 16, 2012 11:28 AM
To: user@hadoop.apache.org
Subject: HDFS using SAN
Hi
I have read scattered documentation across the net which mostly say HDFS
doesn't go well with SAN being used to store data. While
.
** **
Jeff
** **
*From:* Pamecha, Abhishek [mailto:apame...@x.com]
*Sent:* Tuesday, October 16, 2012 11:28 AM
*To:* user@hadoop.apache.org
*Subject:* HDFS using SAN
** **
Hi
** **
I have read scattered documentation across the net which mostly say HDFS
doesn't go well
[mailto:lohit.vijayar...@gmail.com]
Sent: Tuesday, October 16, 2012 3:26 PM
To: user@hadoop.apache.org
Subject: Re: HDFS using SAN
Adding to this. Locality is very important for MapReduce applications. One
might not see much of a difference for small MapReduce jobs running on direct
attached storage vs SAN
15 matches
Mail list logo
