@hadoop.apache.org
Subject: Re: CPU utilization keeps increasing when using HDFS
Because you are using one node Pseudo cluster. When HDFS client write data to
HDFS, client will compute the data chunk checksum and the datanode will verify
it. It costs cpu shares.
You can monitoring the cpu usages for
/sbin/init
>
> 2 root 20 0 00 0 S 0.0 0.0 0:00.02
> [kthreadd]
>
> 3 root RT 0 000 S 0.0 0.0 0:27.01
> [migration/0]
>
>
>
>
>
> *From:* Stanley Shi [mailto:s...@pivotal.io]
> *Sent:* 2014年9月1日 14:32
>
...@pivotal.io]
Sent: 2014年9月1日 14:32
To: user@hadoop.apache.org
Subject: Re: CPU utilization keeps increasing when using HDFS
Would you please give the output of the "top" command? at least to show that
the HDFS process did use that much of CPU;
On Mon, Sep 1, 2014 at 2:19 PM, Sh
Would you please give the output of the "top" command? at least to show
that the HDFS process did use that much of CPU;
On Mon, Sep 1, 2014 at 2:19 PM, Shiyuan Xiao
wrote:
> Hi
>
>
> We have written a MapReduce application based on Hadoop 2.4 which keeps
> reading data from HDFS(Pseudo-distrib
Hi
We have written a MapReduce application based on Hadoop 2.4 which keeps reading
data from HDFS(Pseudo-distributed mode in one node).
And we found the CPU system time and user time of the application keeps
increasing when it is running. If we changed the application to read data from
local di
