On Tue, May 11, 2010 at 8:14 AM, Allen Wittenauer
wrote:
>
> On May 11, 2010, at 7:42 AM, Bastian Lorenz wrote:
>> i have set up an hdfs cluster with three nodes(one master, two
>> datanodes). Everything works fine…so far! Now i want to mount my hdfs
>> with Fuse. This works also. The mountpoint i
As a random, but related aside, there's a nice blog entry on yahoo's dev hadoop
blog page. It's all about throughput and scaling your cluster, etc.
Take care,
-stu
Sent from my Verizon Wireless BlackBerry
-Original Message-
From: Rohan Rai
Date: Fri, 14 May 2010 23:23:00
To: hdfs-use
Jeff,
Which version of hadoop you are using?
You could check following in your setup.
- dfs.http.address configuration on your nn and secondary nn. It is at
this address secondary nn connects to nn.
- check namenode log if you seen an exception at the same time.
I guess this is some configur
In addendum the cluster invokes the max of 44 Maps at a time
Regards
Rohan
Rohan Rai wrote:
Hi Todd
The Node comprises of multi disk (7 to be precise), and there are 6 data
nodes.
The measurement used is that provided by TestDFSIO which comes with
hadoop*test.jar
With the defined block size
Hi Todd
The Node comprises of multi disk (7 to be precise), and there are 6 data
nodes.
The measurement used is that provided by TestDFSIO which comes with
hadoop*test.jar
With the defined block size of 128 MB
44 files of 120 MB was written giving an throughput of 90MB/s
44 files of 100MB gave
Hi Rohan,
How are you measuring throughput? The throughput from a single client will
not scale up as the cluster size increases, as it does not parallelize reads
across multiple nodes. Of course it will also be limited by the inbound
bandwidth of that node.
-Todd
On Fri, May 14, 2010 at 1:23 AM,
I found the problem, I should set parameter dfs.http.address. Seems
the official tutorial has no information for this case.
On Fri, May 14, 2010 at 10:11 AM, Jeff Zhang wrote:
> Hi all,
>
> At first I have the name node and secondary name node on the same
> machine, now I want to move the secon
Hi
Is there a relationship between HDFS Read throught put and Disk Read
throughput.
If yes what would be that.
Lets say we have a disk giving us 120 MB/s
And a Cluster of 6 Nodes
Each Node having 6 disk.
So in an absolutely ideal world it should give us a through put
of 120*6*6 MB/s if used
