org.apache.hadoop.ipc.StandbyException occurs at the thirty of per hour in standby NN
hello, All Installed 2 NN and 3 DN in my hadoop-2.2.0 cluster,and implemented HDFS HA with QJM. Currently, looking at the log of standby NN ,it throws below exception at a regular interval, one hour: 2014-01-24 03:30:01,245 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:ubuntu (auth:SIMPLE) cause:org.apache.hadoop.ipc.StandbyException: Operation category READ is not supported in state standby Actually, the active NN is working ,and no applications access HDFS with error. Does anyone know what the problem is ? Thanks, Francis.Hu
答复: which web ui can track yarn job status?
what the value do you set for yarn.resourcemanager.webapp.address in yarn-site.xml ? Francis.Hu 发件人: ch huang [mailto:justlo...@gmail.com] 发送时间: Wednesday, September 11, 2013 15:34 收件人: user@hadoop.apache.org 主题: which web ui can track yarn job status? hi,all: i use yarn so 50030 is not available for job status,thanks
答复: Is there any way to set Reducer to output to multi-places?
Thanks, Binglin I found the class below that can do it :). http://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapred/lib/MultipleOutputs.html 发件人: Binglin Chang [mailto:decst...@gmail.com] 发送时间: Monday, September 02, 2013 17:37 收件人: user@hadoop.apache.org 主题: Re: Is there any way to set Reducer to output to multi-places? MultipleOutputFormat allows you to write multiple files in one reducer, but can't write output to HDFS and Database concurrently, but I is a good example to show how you can write a customized OutputFormat to achieve this. Please note that for fault tolerance, a reducer may run multiple times, this may generate redundant data, hadoop handles files using FileOutputCommitter, you need to handle database case by yourself(e.g. insert record only if record doesn't exists). On Mon, Sep 2, 2013 at 5:11 PM, Rahul Bhattacharjee wrote: This might help http://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapred/lib/MultipleOutputFormat.html Thanks, Rahul On Mon, Sep 2, 2013 at 2:38 PM, Francis.Hu wrote: hi, All Is there any way to set Reducer to output to multi-places ? For example: a reducer's result can be output to HDFS and Database concurrently. Thanks, Francis.Hu
答复: Is there any way to set Reducer to output to multi-places?
Hi, Rahul I found the class MultipleOutputs which has to do with what I want . It can make Reducers to write to additional outputs(File and Database in my Env.) other than the job default output. http://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapred/lib/MultipleOutputs.html The class comments says below that I want to implement in my project: * * Case one: writing to additional outputs other than the job default output. * * Each additional output, or named output, may be configured with its own * OutputFormat, with its own key class and with its own value * class. * Anyway, thanks! Rahul. Your link leads me to the class MultipleOutputs :). Thanks, Francis.Hu. 发件人: Rahul Bhattacharjee [mailto:rahul.rec@gmail.com] 发送时间: Monday, September 02, 2013 17:12 收件人: user@hadoop.apache.org 主题: Re: Is there any way to set Reducer to output to multi-places? This might help http://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapred/lib/MultipleOutputFormat.html Thanks, Rahul On Mon, Sep 2, 2013 at 2:38 PM, Francis.Hu wrote: hi, All Is there any way to set Reducer to output to multi-places ? For example: a reducer's result can be output to HDFS and Database concurrently. Thanks, Francis.Hu
Is there any way to set Reducer to output to multi-places?
hi, All Is there any way to set Reducer to output to multi-places ? For example: a reducer's result can be output to HDFS and Database concurrently. Thanks, Francis.Hu
答复: Hadoop HA error "JOURNAL is not supported in state standby"
Did you start up your ZKFC service on both of your name nodes ? Thanks, Francis.Hu -邮件原件- 发件人: orahad bigdata [mailto:oracle...@gmail.com] 发送时间: Friday, August 30, 2013 4:09 收件人: user 主题: Hadoop HA error "JOURNAL is not supported in state standby" Hi, I'm facing an error while starting Hadoop in HA(2.0.5) cluster , both the NameNode started in standby mode and not changing the state. When I tried to do health check through "hdfs haadmin -checkhealth " it's giving me below error. Failed on local exception: com.google.protobuf.InvalidProtocolBufferException: Message missing required fields: callId, status; Host Details : local host is: "clone2/XX.XX.XX.XX"; destination host is: "clone1":8020; I checked the logs at NN side. 2013-08-30 00:49:16,074 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hadoop (auth:SIMPLE) cause:org.apache.hadoop.ipc.StandbyException: Operation category JOURNAL is not supported in state standby 2013-08-30 00:49:16,074 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8020, call org.apache.hadoop.hdfs.server.protocol.NamenodeProtocol.rollEditLog from 192.168.126.31:48266: error: org.apache.hadoop.ipc.StandbyException: Operation category JOURNAL is not supported in state standby 2013-08-30 00:49:32,391 INFO org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer: Triggering log roll on remote NameNode clone2:8020 2013-08-30 00:49:32,403 WARN org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer: Unable to trigger a roll of the active NN org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException ): Operation category JOURNAL is not supported in state standby at org.apache.hadoop.hdfs.server.namenode.ha.StandbyState.checkOperation(Standb yState.java:87) at org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.checkOpera tion(NameNode.java:1411) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOperation(FSNamesys tem.java:859) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.rollEditLog(FSNamesystem .java:4445) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.rollEditLog(NameNod eRpcServer.java:766) at org.apache.hadoop.hdfs.protocolPB.NamenodeProtocolServerSideTranslatorPB.rol lEditLog(NamenodeProtocolServerSideTranslatorPB.java:139) at org.apache.hadoop.hdfs.protocol.proto.NamenodeProtocolProtos$NamenodeProtoco lService$2.callBlockingMethod(NamenodeProtocolProtos.java:8758) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(Proto bufRpcEngine.java:454) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1014) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1741) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1737) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja va:1478) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1735) at org.apache.hadoop.ipc.Client.call(Client.java:1235) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.jav a:202) at $Proxy11.rollEditLog(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.NamenodeProtocolTranslatorPB.rollEditLog(N amenodeProtocolTranslatorPB.java:139) at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.triggerActiveLogRoll (EditLogTailer.java:268) at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.access$600(EditLogTa iler.java:61) at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread. doWork(EditLogTailer.java:310) at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread. access$200(EditLogTailer.java:279) at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$ 1.run(EditLogTailer.java:296) at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.ja va:456) at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread. run(EditLogTailer.java:292) Did I missed something? Thanks
答复: about append
it is due to that it cannot handle the situation well about the concurrent access to same file. 发件人: gsw204 [mailto:gsw...@163.com] 发送时间: Tuesday, August 20, 2013 18:28 收件人: user 主题: about append hi, I want to know why append does not support in hadoop-1.1.x? _ gsw
Cannot kill a job with correct job ID.
Hi, All
In my resources manager web page, I see the job
application_1376968101508_0009 is running.
I want to kill the job with the command: bin/mapred job -kill
application_1376968101508_0009
It throws below error at all the time :
Exception in thread "main" java.lang.IllegalArgumentException: JobId string
: application_1376968101508_0009 is not properly formed
at org.apache.hadoop.mapreduce.JobID.forName(JobID.java:156)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:276)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1199)
And then looked at the source code of JobID class, found below method :
JobID.forName(String).
It means we need to kill a job with name "job_1376968101508_0009", not
"application_1376968101508_0009".
That is really strange to kill a job with special name and without the
actual job name showing in resources manager UI.
Is this an existing issue ?
protected static final String JOB = "job";
public static JobID forName(String str) throws IllegalArgumentException {
if(str == null)
return null;
try {
String[] parts = str.split("_");
if(parts.length == 3) {
if(parts[0].equals(JOB)) {
return new org.apache.hadoop.mapred.JobID(parts[1],
Integer.parseInt(parts[2]));
}
}
}catch (Exception ex) {//fall below
}
throw new IllegalArgumentException("JobId string : " + str
+ " is not properly formed");
}
Thanks,
Francis.Hu
答复: Passing an object in mapper
hi,
what I did when data need to be saved into DB :
//configure DB connection info.
DBConfiguration.configureDB(config,
HadoopJobConfigParser.getProperty(DB_DRIVER),
HadoopJobConfigParser.getProperty(DB_URL),
HadoopJobConfigParser.getProperty(DB_USER),
HadoopJobConfigParser.getProperty(DB_PWD));
//write a OutputKeyClass like below:
public class DisplayRequestRecord implements DBWritable,
WritableComparable{
//TODO implement write() and readFiels() methods
}
//set the job's output
job.setOutputKeyClass(DisplayRequestRecord.class);
job.setOutputValueClass(DisplayRequestRecord.class);
DBOutputFormat.setOutput(job, tableName(),Fields());
//define reducer class like below:
public class DBDisplayRequestReducerNew extends Reducer{
public void reduce(Text key, Iterable values,
Context context) throws IOException, InterruptedException {
DisplayRequestRecord dr = new DisplayRequestRecord();
context.write(dr,dr);
}
}
Hope it helps!
Thanks,
Francis.Hu
发件人: Sivaram RL [mailto:sivaram...@gmail.com]
发送时间: Thursday, August 15, 2013 11:26
收件人: user@hadoop.apache.org
主题: Re: Passing an object in mapper
Hi,
In the Configuration object in your driver class you can set the properties
as key value pair. This configuration object will be set in the Job Object.
The same properties can be accessed in the mapper/reducer using the Context
Object -> getConfiguration() -> get(propertyName).
Hope this helps.
Regards,
Sivaram R L
On Thu, Aug 15, 2013 at 8:39 AM, jamal sasha wrote:
Hi,
I am initializing an object in driver code.
For sake of argument let say I want to save data to some database..
say:
Connection con = new Connection(host, db);
Now, in reducer I want to do something like
con.write(key,value)
So, how do i pass this object from driver to mapper / reducer/?
Any clue? suggestions?
Thanks
答复: cannot submit a job via java client in hadoop- 2.0.5-alpha
Hi, Devaraj k and Azuryy Yu
Thanks both of you.
I just get it resolved. The problem is that below highlighted jar is not
included in my java client side so that when the Job is initializing, it can
not find the class YarnClientProtocolProvider to do further initialization.
Then it causes the exception in Cluster.initialize() is thrown out.
hadoop-mapreduce-client-jobclient-2.0.5-alpha.jar
Thanks,
Francis.Hu
发件人: Devaraj k [mailto:devara...@huawei.com]
发送时间: Wednesday, July 10, 2013 19:11
收件人: user@hadoop.apache.org
主题: RE: cannot submit a job via java client in hadoop- 2.0.5-alpha
Hi Francis,
Could you check whether those configuration files are
getting loaded or not, There could be a chance that these configuration
files are not getting loaded into configuration object due to some invalid
path reason.
conf1.addResource("resources/mapred-site.xml");
// Can you try printing the properties
‘yarn.resourcemanager.address’ & ‘mapreduce.framework.name’ values and
check they are coming as per the config files
JobConf conf = new JobConf(conf1);
Thanks
Devaraj k
From: Azuryy Yu [mailto:azury...@gmail.com]
Sent: 10 July 2013 16:22
To: user@hadoop.apache.org
Subject: Re: cannot submit a job via java client in hadoop- 2.0.5-alpha
you didn't set yarn.nodemanager.address in your yarn-site.xml
On Wed, Jul 10, 2013 at 4:33 PM, Francis.Hu
wrote:
Hi,All
I have a hadoop- 2.0.5-alpha cluster with 3 data nodes . I have Resource
Manager and all data nodes started and can access web ui of Resource
Manager.
I wrote a java client to submit a job as TestJob class below. But the job is
never submitted successfully. It throws out exception all the time.
My configurations are attached. Can anyone help me? Thanks.
-my-java client
public class TestJob {
public void execute() {
Configuration conf1 = new Configuration();
conf1.addResource("resources/core-site.xml");
conf1.addResource("resources/hdfs-site.xml");
conf1.addResource("resources/yarn-site.xml");
conf1.addResource("resources/mapred-site.xml");
JobConf conf = new JobConf(conf1);
conf.setJar("/home/francis/hadoop-jobs/MapReduceJob.jar");
conf.setJobName("Test");
conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(TextOutputFormat.class);
conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(IntWritable.class);
conf.setMapperClass(DisplayRequestMapper.class);
conf.setReducerClass(DisplayRequestReducer.class);
FileInputFormat.setInputPaths(conf,new
Path("/home/francis/hadoop-jobs/2013070907.FNODE.2.txt"));
FileOutputFormat.setOutputPath(conf, new
Path("/home/francis/hadoop-jobs/result/"));
try {
JobClient client = new JobClient(conf);
RunningJob job = client.submitJob(conf);
job.waitForCompletion();
} catch (IOException e) {
e.printStackTrace();
}
}
}
--Exception
jvm 1| java.io.IOException: Cannot initialize Cluster. Please check your
configuration for mapreduce.framework.name and the correspond server
addresses.
jvm 1| at
org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:119)
jvm 1| at
org.apache.hadoop.mapreduce.Cluster.(Cluster.java:81)
jvm 1| at
org.apache.hadoop.mapreduce.Cluster.(Cluster.java:74)
jvm 1| at
org.apache.hadoop.mapred.JobClient.init(JobClient.java:482)
jvm 1| at
org.apache.hadoop.mapred.JobClient.(JobClient.java:461)
jvm 1| at
com.rh.elastic.hadoop.job.TestJob.execute(TestJob.java:59)
Thanks,
Francis.Hu
答复: cannot submit a job via java client in hadoop- 2.0.5-alpha
Actually ,I have mapreduce.framework.name configured in mapred-site.xml, see
below:
mapreduce.framework.name
yarn
Execution framework set to Hadoop YARN.
发件人: hadoop hive [mailto:hadooph...@gmail.com]
发送时间: Wednesday, July 10, 2013 18:39
收件人: user@hadoop.apache.org
主题: Re: cannot submit a job via java client in hadoop- 2.0.5-alpha
Here its showing like you are not using mapreduce.framework.name as yarn,
please resend it we are unable to see the configuration
On Wed, Jul 10, 2013 at 1:33 AM, Francis.Hu
wrote:
Hi,All
I have a hadoop- 2.0.5-alpha cluster with 3 data nodes . I have Resource
Manager and all data nodes started and can access web ui of Resource
Manager.
I wrote a java client to submit a job as TestJob class below. But the job is
never submitted successfully. It throws out exception all the time.
My configurations are attached. Can anyone help me? Thanks.
-my-java client
public class TestJob {
public void execute() {
Configuration conf1 = new Configuration();
conf1.addResource("resources/core-site.xml");
conf1.addResource("resources/hdfs-site.xml");
conf1.addResource("resources/yarn-site.xml");
conf1.addResource("resources/mapred-site.xml");
JobConf conf = new JobConf(conf1);
conf.setJar("/home/francis/hadoop-jobs/MapReduceJob.jar");
conf.setJobName("Test");
conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(TextOutputFormat.class);
conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(IntWritable.class);
conf.setMapperClass(DisplayRequestMapper.class);
conf.setReducerClass(DisplayRequestReducer.class);
FileInputFormat.setInputPaths(conf,new
Path("/home/francis/hadoop-jobs/2013070907.FNODE.2.txt"));
FileOutputFormat.setOutputPath(conf, new
Path("/home/francis/hadoop-jobs/result/"));
try {
JobClient client = new JobClient(conf);
RunningJob job = client.submitJob(conf);
job.waitForCompletion();
} catch (IOException e) {
e.printStackTrace();
}
}
}
--Exception
jvm 1| java.io.IOException: Cannot initialize Cluster. Please check your
configuration for mapreduce.framework.name and the correspond server
addresses.
jvm 1| at
org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:119)
jvm 1| at
org.apache.hadoop.mapreduce.Cluster.(Cluster.java:81)
jvm 1| at
org.apache.hadoop.mapreduce.Cluster.(Cluster.java:74)
jvm 1| at
org.apache.hadoop.mapred.JobClient.init(JobClient.java:482)
jvm 1| at
org.apache.hadoop.mapred.JobClient.(JobClient.java:461)
jvm 1| at
com.rh.elastic.hadoop.job.TestJob.execute(TestJob.java:59)
Thanks,
Francis.Hu
cannot submit a job via java client in hadoop- 2.0.5-alpha
Hi,All
I have a hadoop- 2.0.5-alpha cluster with 3 data nodes . I have Resource
Manager and all data nodes started and can access web ui of Resource
Manager.
I wrote a java client to submit a job as TestJob class below. But the job is
never submitted successfully. It throws out exception all the time.
My configurations are attached. Can anyone help me? Thanks.
-my-java client
public class TestJob {
public void execute() {
Configuration conf1 = new Configuration();
conf1.addResource("resources/core-site.xml");
conf1.addResource("resources/hdfs-site.xml");
conf1.addResource("resources/yarn-site.xml");
conf1.addResource("resources/mapred-site.xml");
JobConf conf = new JobConf(conf1);
conf.setJar("/home/francis/hadoop-jobs/MapReduceJob.jar");
conf.setJobName("Test");
conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(TextOutputFormat.class);
conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(IntWritable.class);
conf.setMapperClass(DisplayRequestMapper.class);
conf.setReducerClass(DisplayRequestReducer.class);
FileInputFormat.setInputPaths(conf,new
Path("/home/francis/hadoop-jobs/2013070907.FNODE.2.txt"));
FileOutputFormat.setOutputPath(conf, new
Path("/home/francis/hadoop-jobs/result/"));
try {
JobClient client = new JobClient(conf);
RunningJob job = client.submitJob(conf);
job.waitForCompletion();
} catch (IOException e) {
e.printStackTrace();
}
}
}
--Exception
jvm 1| java.io.IOException: Cannot initialize Cluster. Please check your
configuration for mapreduce.framework.name and the correspond server
addresses.
jvm 1| at
org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:119)
jvm 1| at
org.apache.hadoop.mapreduce.Cluster.(Cluster.java:81)
jvm 1| at
org.apache.hadoop.mapreduce.Cluster.(Cluster.java:74)
jvm 1| at
org.apache.hadoop.mapred.JobClient.init(JobClient.java:482)
jvm 1| at
org.apache.hadoop.mapred.JobClient.(JobClient.java:461)
jvm 1| at
com.rh.elastic.hadoop.job.TestJob.execute(TestJob.java:59)
Thanks,
Francis.Hu
yarn.acl.enable
false
Enable ACLs? Defaults to false.
yarn.resourcemanager.address
192.168.219.129:9001
ResourceManager host:port for clients to submit jobs.
yarn.resourcemanager.scheduler.address
192.168.219.129:8030
ResourceManager host:port for ApplicationMasters to talk to Scheduler to obtain resources.
yarn.resourcemanager.resource-tracker.address
192.168.219.129:8031
ResourceManager host:port for NodeManagers.
yarn.resourcemanager.admin.address
192.168.219.129:8033
ResourceManager host:port for administrative commands.
yarn.resourcemanager.webapp.address
192.168.219.129:8088
ResourceManager web-ui host:port.
yarn.resourcemanager.scheduler.class
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
ResourceManager Scheduler class.
yarn.scheduler.minimum-allocation-mb
1024
Minimum limit of memory to allocate to each container request at the Resource Manager.
yarn.scheduler.maximum-allocation-mb
8192
Maximum limit of memory to allocate to each container request at the Resource Manager.
yarn.nodemanager.resource.memory-mb
8192
Resource i.e. available physical memory, in MB, for given NodeManager.Defines total available resources on the NodeManager to be made available to running containers
yarn.nodemanager.vmem-pmem-ratio
2.1
Maximum ratio by which virtual memory usage of tasks may exceed physical memory
yarn.nodemanager.local-dirs
/home/francis/hadoop2-hdfs/yarn
Comma-separated list of paths on the local filesystem where intermediate data is written.Multiple paths help spread disk i/o.
yarn.nodemanager.log-dirs
/home/francis/hadoop2-hdfs/yarn-log
Comma-separated list of paths on the local filesystem where logs are written.Multiple paths help spread disk i/o.
yarn.nodemanager.log.retain-seconds
10800
Default time (in seconds) to retain log files on the NodeManager Only applicable if log-aggregation is disabled.
yarn.nodemanager.remote-app-log-dir
/logs
HDFS directory where the application logs are moved on application completion. Need to set appropriate permissions. Only applicable if log-aggregation is enabled.
yarn.nodemanager.remote-app-log-dir-suffix
logs
Suffix appended to the remote log dir. Logs will be aggregated to ${yarn.nodemanager.remote-app-log-dir}/${user}/${thisParam} Only applicable if log-aggregation is enabled.
yarn.nodemanager.aux-services
mapreduce.shuffle
Shuffle service that needs to be set for Map Reduce applications.
yarn.log-aggregation.retain-seconds
-1
How long to keep aggregation logs before deleting them. -1 disables. Be c
答复: Decomssion datanode - no response
I know the default value is 10 minutes and 30 seconds for switching datanodes from live to dead. 发件人: Azuryy Yu [mailto:azury...@gmail.com] 发送时间: Friday, July 05, 2013 10:42 收件人: user@hadoop.apache.org 主题: Decomssion datanode - no response Hi, I am using hadoop-2.0.5-alpha, and I added 5 datanodes into dfs_exclude, hdfs-site.xml: dfs.hosts.exclude /usr/local/hadoop/conf/dfs_exclude then: hdfs dfsadmin -refreshNodes but there is no decomssion nodes showed on the webUI. and not any releated logs in the datanode log. what's wrong?
答复: interval from Live to Dead about DataNode
Nicolas, Thanks for your help. had a look at HDFS-3703. So I need to turn on dfs.namenode.check.stale.datanode and set a shorter time for dfs.namenode.stale.datanode.interval Thanks, Francis.Hu 发件人: Nicolas Liochon [mailto:nkey...@gmail.com] 发送时间: Thursday, July 04, 2013 16:28 收件人: user@hadoop.apache.org 主题: Re: interval from Live to Dead about DataNode Hi, It's 10 minutes and 30s. See the stale mode described in HDFS-3703 if you need something shorter. Cheers, Nicolas On Thu, Jul 4, 2013 at 10:05 AM, Francis.Hu wrote: Hi,All Does anyone know when the status of datanode switch from live to dead at the internal of namenode ? the scenario: When i stopped a datanode with command, the status of that datanode in the web UI of namenode displays 'live' and 'In Service' for almost 5 minutes. I know the default value of datanode dfs.hreadbeat.interval is set as 3 seconds and dfs.namenode.decommission.interval is set 30 seconds. Thanks, Francis.Hu
interval from Live to Dead about DataNode
Hi,All Does anyone know when the status of datanode switch from live to dead at the internal of namenode ? the scenario: When i stopped a datanode with command, the status of that datanode in the web UI of namenode displays 'live' and 'In Service' for almost 5 minutes. I know the default value of datanode dfs.hreadbeat.interval is set as 3 seconds and dfs.namenode.decommission.interval is set 30 seconds. Thanks, Francis.Hu
reply: a question about dfs.replication
YouPeng Yang, you said that may be the answer. Thank you. 发件人: YouPeng Yang [mailto:yypvsxf19870...@gmail.com] 发送时间: Tuesday, July 02, 2013 12:52 收件人: user@hadoop.apache.org 主题: Re: reply: a question about dfs.replication HI HU and Yu Aggree with dfs.replication is a client side configuration, not server side. It make the point in my last mail sense. And the cmd:hdfs dfs -setrep -R -w 2 / solve the problem that I can not change the existed file's replication value. 2013/7/2 Azuryy Yu It's not HDFS issue. dfs.replication is a client side configuration, not server side. so you need to set it to '2' on your client side( your application running on). THEN execute command such as : hdfs dfs -put or call HDFS API in java application. On Tue, Jul 2, 2013 at 12:25 PM, Francis.Hu wrote: Thanks all of you, I just get the problem fixed through the command: hdfs dfs -setrep -R -w 2 / Is that an issue of HDFS ? Why do i need to execute manually a command to tell the hadoop the replication factor even it is set in hdfs-site.xml ? Thanks, Francis.Hu 发件人: Francis.Hu [mailto:francis...@reachjunction.com] 发送时间: Tuesday, July 02, 2013 11:30 收件人: user@hadoop.apache.org 主题: 答复: 答复: a question about dfs.replication Yes , it returns 2 correctly after "hdfs getconf -confkey dfs.replication" but in web page ,it is 3 as below: 发件人: yypvsxf19870706 [mailto:yypvsxf19870...@gmail.com] 发送时间: Monday, July 01, 2013 23:24 收件人: user@hadoop.apache.org 主题: Re: 答复: a question about dfs.replication Hi Could you please get the property value by using : hdfs getconf -confkey dfs.replication. 鍙戣嚜鎴戠殑 iPhone 鍦?2013-7-1锛?5:51锛孎rancis.Hu 鍐欓亾锛?br> Actually, My java client is running with the same configuration as the hadoop's . The dfs.replication is already set as 2 in my hadoop's configuration. So i think the dfs.replication is already overrided by my configuration in hdfs-site.xml. but seems it doesn't work even i overrided the parameter evidently. 鍙戜欢浜?span lang="EN-US">: 袝屑械谢褜褟薪芯胁 袘芯褉懈褋 [mailto:emelya...@post.km.ru] 鍙戦€佹椂闂?span lang="EN-US">: Monday, July 01, 2013 15:18 鏀朵欢浜?span lang="EN-US">: user@hadoop.apache.org 涓婚: Re: a question about dfs.replication On 01.07.2013 10:19, Francis.Hu wrote: Hi, All I am installing a cluster with Hadoop 2.0.5-alpha. I have one namenode and two datanodes. The dfs.replication is set as 2 in hdfs-site.xml. After all configuration work is done, I started all nodes. Then I saved a file into HDFS through java client. nOW I can access hdfs web page: x.x.x.x:50070,and also see the file is already listed in the hdfs list. My question is: The replication column in HDFS web page is showing as 3, not 2. Does anyone know What the problem is? ---Actual setting of hdfs-site.xml dfs.replication 2 After that, I typed dfsamdin command to check the file: hdfs fsck /test3/ The result of above command: /test3/hello005.txt: Under replicated BP-609310498-192.168.219.129-1372323727200:blk_-1069303317294683372_1006. Target Replicas is 3 but found 2 replica(s). Status: HEALTHY Total size:35 B Total dirs:1 Total files: 1 Total blocks (validated): 1 (avg. block size 35 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks:0 (0.0 %) Under-replicated blocks: 1 (100.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor:2 Average block replication: 2.0 Corrupt blocks:0 Missing replicas: 1 (33.32 %) Number of data-nodes: 3 Number of racks: 1 FSCK ended at Sat Jun 29 16:51:37 CST 2013 in 6 milliseconds Thanks, Francis Hu If I'm not mistaking "dfs.replication" parameter in config sets only default replication factor, which can be overrided when putting file to hdfs. <><>
reply: a question about dfs.replication
actually, my client side is already set to "2". 发件人: Azuryy Yu [mailto:azury...@gmail.com] 发送时间: Tuesday, July 02, 2013 12:40 收件人: user@hadoop.apache.org 主题: Re: reply: a question about dfs.replication It's not HDFS issue. dfs.replication is a client side configuration, not server side. so you need to set it to '2' on your client side( your application running on). THEN execute command such as : hdfs dfs -put or call HDFS API in java application. On Tue, Jul 2, 2013 at 12:25 PM, Francis.Hu wrote: Thanks all of you, I just get the problem fixed through the command: hdfs dfs -setrep -R -w 2 / Is that an issue of HDFS ? Why do i need to execute manually a command to tell the hadoop the replication factor even it is set in hdfs-site.xml ? Thanks, Francis.Hu 发件人: Francis.Hu [mailto:francis...@reachjunction.com] 发送时间: Tuesday, July 02, 2013 11:30 收件人: user@hadoop.apache.org 主题: 答复: 答复: a question about dfs.replication Yes , it returns 2 correctly after "hdfs getconf -confkey dfs.replication" but in web page ,it is 3 as below: 发件人: yypvsxf19870706 [mailto:yypvsxf19870...@gmail.com] 发送时间: Monday, July 01, 2013 23:24 收件人: user@hadoop.apache.org 主题: Re: 答复: a question about dfs.replication Hi Could you please get the property value by using : hdfs getconf -confkey dfs.replication. 鍙戣嚜鎴戠殑 iPhone 鍦?2013-7-1锛?5:51锛孎rancis.Hu 鍐欓亾锛?br> Actually, My java client is running with the same configuration as the hadoop's . The dfs.replication is already set as 2 in my hadoop's configuration. So i think the dfs.replication is already overrided by my configuration in hdfs-site.xml. but seems it doesn't work even i overrided the parameter evidently. 鍙戜欢浜?span lang="EN-US">: 袝屑械谢褜褟薪芯胁 袘芯褉懈褋 [mailto:emelya...@post.km.ru] 鍙戦€佹椂闂?span lang="EN-US">: Monday, July 01, 2013 15:18 鏀朵欢浜?span lang="EN-US">: user@hadoop.apache.org 涓婚: Re: a question about dfs.replication On 01.07.2013 10:19, Francis.Hu wrote: Hi, All I am installing a cluster with Hadoop 2.0.5-alpha. I have one namenode and two datanodes. The dfs.replication is set as 2 in hdfs-site.xml. After all configuration work is done, I started all nodes. Then I saved a file into HDFS through java client. nOW I can access hdfs web page: x.x.x.x:50070,and also see the file is already listed in the hdfs list. My question is: The replication column in HDFS web page is showing as 3, not 2. Does anyone know What the problem is? ---Actual setting of hdfs-site.xml dfs.replication 2 After that, I typed dfsamdin command to check the file: hdfs fsck /test3/ The result of above command: /test3/hello005.txt: Under replicated BP-609310498-192.168.219.129-1372323727200:blk_-1069303317294683372_1006. Target Replicas is 3 but found 2 replica(s). Status: HEALTHY Total size:35 B Total dirs:1 Total files: 1 Total blocks (validated): 1 (avg. block size 35 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks:0 (0.0 %) Under-replicated blocks: 1 (100.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor:2 Average block replication: 2.0 Corrupt blocks:0 Missing replicas: 1 (33.32 %) Number of data-nodes: 3 Number of racks: 1 FSCK ended at Sat Jun 29 16:51:37 CST 2013 in 6 milliseconds Thanks, Francis Hu If I'm not mistaking "dfs.replication" parameter in config sets only default replication factor, which can be overrided when putting file to hdfs. <><>
reply: a question about dfs.replication
Thanks all of you, I just get the problem fixed through the command: hdfs dfs -setrep -R -w 2 / Is that an issue of HDFS ? Why do i need to execute manually a command to tell the hadoop the replication factor even it is set in hdfs-site.xml ? Thanks, Francis.Hu 发件人: Francis.Hu [mailto:francis...@reachjunction.com] 发送时间: Tuesday, July 02, 2013 11:30 收件人: user@hadoop.apache.org 主题: 答复: 答复: a question about dfs.replication Yes , it returns 2 correctly after "hdfs getconf -confkey dfs.replication" but in web page ,it is 3 as below: 发件人: yypvsxf19870706 [mailto:yypvsxf19870...@gmail.com] 发送时间: Monday, July 01, 2013 23:24 收件人: user@hadoop.apache.org 主题: Re: 答复: a question about dfs.replication Hi Could you please get the property value by using : hdfs getconf -confkey dfs.replication. 鍙戣嚜鎴戠殑 iPhone 鍦?2013-7-1锛?5:51锛孎rancis.Hu 鍐欓亾锛?br> Actually, My java client is running with the same configuration as the hadoop's . The dfs.replication is already set as 2 in my hadoop's configuration. So i think the dfs.replication is already overrided by my configuration in hdfs-site.xml. but seems it doesn't work even i overrided the parameter evidently. 鍙戜欢浜?span lang="EN-US">: 袝屑械谢褜褟薪芯胁 袘芯褉懈褋 [mailto:emelya...@post.km.ru] 鍙戦€佹椂闂?span lang="EN-US">: Monday, July 01, 2013 15:18 鏀朵欢浜?span lang="EN-US">: user@hadoop.apache.org 涓婚: Re: a question about dfs.replication On 01.07.2013 10:19, Francis.Hu wrote: Hi, All I am installing a cluster with Hadoop 2.0.5-alpha. I have one namenode and two datanodes. The dfs.replication is set as 2 in hdfs-site.xml. After all configuration work is done, I started all nodes. Then I saved a file into HDFS through java client. nOW I can access hdfs web page: x.x.x.x:50070,and also see the file is already listed in the hdfs list. My question is: The replication column in HDFS web page is showing as 3, not 2. Does anyone know What the problem is? ---Actual setting of hdfs-site.xml dfs.replication 2 After that, I typed dfsamdin command to check the file: hdfs fsck /test3/ The result of above command: /test3/hello005.txt: Under replicated BP-609310498-192.168.219.129-1372323727200:blk_-1069303317294683372_1006. Target Replicas is 3 but found 2 replica(s). Status: HEALTHY Total size:35 B Total dirs:1 Total files: 1 Total blocks (validated): 1 (avg. block size 35 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks:0 (0.0 %) Under-replicated blocks: 1 (100.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor:2 Average block replication: 2.0 Corrupt blocks:0 Missing replicas: 1 (33.32 %) Number of data-nodes: 3 Number of racks: 1 FSCK ended at Sat Jun 29 16:51:37 CST 2013 in 6 milliseconds Thanks, Francis Hu If I'm not mistaking "dfs.replication" parameter in config sets only default replication factor, which can be overrided when putting file to hdfs. <><>
答复: 答复: a question about dfs.replication
Yes , it returns 2 correctly after "hdfs getconf -confkey dfs.replication" but in web page ,it is 3 as below: 发件人: yypvsxf19870706 [mailto:yypvsxf19870...@gmail.com] 发送时间: Monday, July 01, 2013 23:24 收件人: user@hadoop.apache.org 主题: Re: 答复: a question about dfs.replication Hi Could you please get the property value by using : hdfs getconf -confkey dfs.replication. 鍙戣嚜鎴戠殑 iPhone 鍦?2013-7-1锛?5:51锛孎rancis.Hu 鍐欓亾锛?br> Actually, My java client is running with the same configuration as the hadoop's . The dfs.replication is already set as 2 in my hadoop's configuration. So i think the dfs.replication is already overrided by my configuration in hdfs-site.xml. but seems it doesn't work even i overrided the parameter evidently. 鍙戜欢浜?span lang="EN-US">: 袝屑械谢褜褟薪芯胁 袘芯褉懈褋 [mailto:emelya...@post.km.ru] 鍙戦€佹椂闂?span lang="EN-US">: Monday, July 01, 2013 15:18 鏀朵欢浜?span lang="EN-US">: user@hadoop.apache.org 涓婚: Re: a question about dfs.replication On 01.07.2013 10:19, Francis.Hu wrote: Hi, All I am installing a cluster with Hadoop 2.0.5-alpha. I have one namenode and two datanodes. The dfs.replication is set as 2 in hdfs-site.xml. After all configuration work is done, I started all nodes. Then I saved a file into HDFS through java client. nOW I can access hdfs web page: x.x.x.x:50070,and also see the file is already listed in the hdfs list. My question is: The replication column in HDFS web page is showing as 3, not 2. Does anyone know What the problem is? ---Actual setting of hdfs-site.xml dfs.replication 2 After that, I typed dfsamdin command to check the file: hdfs fsck /test3/ The result of above command: /test3/hello005.txt: Under replicated BP-609310498-192.168.219.129-1372323727200:blk_-1069303317294683372_1006. Target Replicas is 3 but found 2 replica(s). Status: HEALTHY Total size:35 B Total dirs:1 Total files: 1 Total blocks (validated): 1 (avg. block size 35 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks:0 (0.0 %) Under-replicated blocks: 1 (100.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor:2 Average block replication: 2.0 Corrupt blocks:0 Missing replicas: 1 (33.32 %) Number of data-nodes: 3 Number of racks: 1 FSCK ended at Sat Jun 29 16:51:37 CST 2013 in 6 milliseconds Thanks, Francis Hu If I'm not mistaking "dfs.replication" parameter in config sets only default replication factor, which can be overrided when putting file to hdfs. <><>
答复: a question about dfs.replication
Actually, My java client is running with the same configuration as the hadoop's . The dfs.replication is already set as 2 in my hadoop's configuration. So i think the dfs.replication is already overrided by my configuration in hdfs-site.xml. but seems it doesn't work even i overrided the parameter evidently. 发件人: Емельянов Борис [mailto:emelya...@post.km.ru] 发送时间: Monday, July 01, 2013 15:18 收件人: user@hadoop.apache.org 主题: Re: a question about dfs.replication On 01.07.2013 10:19, Francis.Hu wrote: Hi, All I am installing a cluster with Hadoop 2.0.5-alpha. I have one namenode and two datanodes. The dfs.replication is set as 2 in hdfs-site.xml. After all configuration work is done, I started all nodes. Then I saved a file into HDFS through java client. nOW I can access hdfs web page: x.x.x.x:50070,and also see the file is already listed in the hdfs list. My question is: The replication column in HDFS web page is showing as 3, not 2. Does anyone know What the problem is? ---Actual setting of hdfs-site.xml dfs.replication 2 After that, I typed dfsamdin command to check the file: hdfs fsck /test3/ The result of above command: /test3/hello005.txt: Under replicated BP-609310498-192.168.219.129-1372323727200:blk_-1069303317294683372_1006. Target Replicas is 3 but found 2 replica(s). Status: HEALTHY Total size:35 B Total dirs:1 Total files: 1 Total blocks (validated): 1 (avg. block size 35 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks:0 (0.0 %) Under-replicated blocks: 1 (100.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor:2 Average block replication: 2.0 Corrupt blocks:0 Missing replicas: 1 (33.32 %) Number of data-nodes: 3 Number of racks: 1 FSCK ended at Sat Jun 29 16:51:37 CST 2013 in 6 milliseconds Thanks, Francis Hu If I'm not mistaking "dfs.replication" parameter in config sets only default replication factor, which can be overrided when putting file to hdfs.
a question about dfs.replication
Hi, All I am installing a cluster with Hadoop 2.0.5-alpha. I have one namenode and two datanodes. The dfs.replication is set as 2 in hdfs-site.xml. After all configuration work is done, I started all nodes. Then I saved a file into HDFS through java client. nOW I can access hdfs web page: x.x.x.x:50070,and also see the file is already listed in the hdfs list. My question is: The replication column in HDFS web page is showing as 3, not 2. Does anyone know What the problem is? ---Actual setting of hdfs-site.xml dfs.replication 2 After that, I typed dfsamdin command to check the file: hdfs fsck /test3/ The result of above command: /test3/hello005.txt: Under replicated BP-609310498-192.168.219.129-1372323727200:blk_-1069303317294683372_1006. Target Replicas is 3 but found 2 replica(s). Status: HEALTHY Total size:35 B Total dirs:1 Total files: 1 Total blocks (validated): 1 (avg. block size 35 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks:0 (0.0 %) Under-replicated blocks: 1 (100.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor:2 Average block replication: 2.0 Corrupt blocks:0 Missing replicas: 1 (33.32 %) Number of data-nodes: 3 Number of racks: 1 FSCK ended at Sat Jun 29 16:51:37 CST 2013 in 6 milliseconds Thanks, Francis Hu
