What Shumin told is correct,hadoop configurations has been over written
through client application.
We have faced similar type of issue,Where default replication factor was
mentioned 2 in hadoop configuration.But when when ever the client
application writes a files,it was having 3 copies in
Replication, block size, etc. are all per-file and pure client
supplied properties. They either take their default from the client
config, or directly from an API argument override.
On Sun, Jul 14, 2013 at 4:14 PM, varun kumar varun@gmail.com wrote:
What Shumin told is correct,hadoop
Check out how does the writing happens on HDFS...
When client issues the command hadoop fs -put local_source
hdfs_destination, client contact the namenode that he wants to write and
then NameNode creates a blockID and ask three datanodes ( if replication on
the client side is set to 3) to host
Hi Please help me on this. Its urgent.
From: Shalish VJ shalis...@yahoo.com
To: hadoop-mailerlist user@hadoop.apache.org
Sent: Friday, July 12, 2013 10:20 PM
Subject: Hadoop property precedence
Hi,
Suppose block size set in configuration file
Shalish,
The default block size is 64MB which is good at the client end. Make sure
the same at your end also in conf. You can increase the size of each block
to 128MB or greater than that only thing you can see the processing will be
fast but at end there may be chances of losing data.
Thanks,
the conf that client running on will take effect.
On Jul 13, 2013 4:42 PM, Kiran Dangeti kirandkumar2...@gmail.com wrote:
Shalish,
The default block size is 64MB which is good at the client end. Make sure
the same at your end also in conf. You can increase the size of each block
to 128MB or
Hi,
Ru sure?
or have u ever tried it out.
Pls advice.
From: Azuryy Yu azury...@gmail.com
To: user@hadoop.apache.org
Sent: Saturday, July 13, 2013 3:19 PM
Subject: Re: Hadoop property precedence
the conf that client running on will take effect.
On Jul
sure?
or have u ever tried it out.
Pls advice.
From: Azuryy Yu azury...@gmail.com
To: user@hadoop.apache.org
Sent: Saturday, July 13, 2013 3:19 PM
Subject: Re: Hadoop property precedence
the conf that client running on will take effect.
On Jul 13, 2013 4:42 PM, Kiran Dangeti kirandkumar2
I Think the client side configuration will take effect.
Shumin
On Jul 12, 2013 11:50 AM, Shalish VJ shalis...@yahoo.com wrote:
Hi,
Suppose block size set in configuration file at client side is 64MB,
block size set in configuration file at name node side is 128MB and block
size set in
Hi,
Suppose block size set in configuration file at client side is 64MB,
block size set in configuration file at name node side is 128MB and block size
set in configuration file at datanode side is something else.
Please advice, If the client is writing a file to hdfs,which property would
10 matches
Mail list logo
