Re: Set yarn.nodemanager.resource.memory-mb higher than node physical memory

[Jeff Hubbs]

To make sure I understand...you've allocated /ten times/ your physical 
RAM for containers? If so, I think that's your issue.


For reference, under Hadoop 3.x I didn't have a cluster that would 
really do anything until its worker nodes had at least 8GiB.


On 8/14/19 12:10 PM, . . wrote:

Hi all,

I installed a basic 3 nodes Hadoop 2.9.1 cluster and playing with YARN 
settings.

The 3 nodes has following configuration:
1 cpu / 1 core?? / 512MB RAM

I wonder I was able to configure yarn-site.xml with following settings 
(higher than node physical limits) and successfully run a mapreduce 
'pi 1 10' job


quote...
?? 
yarn.resourcemanager.scheduler.classorg.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler


?? ?? 
yarn.nodemanager.resource.memory-mb
?? ?? ?? ?? 5120
?? ?? ?? ?? Amount of physical memory, in MB, that can be 
allocated for containers. If set to -1 and 
yarn.nodemanager.resource.detect-hardware-capabilities is true, it is 
automatically calculated. In other cases, the default is 
8192MB

?? ?? 

?? ?? 
yarn.nodemanager.resource.cpu-vcores
?? ?? ?? ?? 6
?? ?? ?? ?? Number of CPU cores that can be allocated for 
containers.

?? ?? 
...unquote

Can anyone provide an explanation please?

Should 'yarn.nodemanager.vmem-check-enabled' and 
'yarn.nodemanager.pmem-check-enabled' properties (set to 'true' as 
default) check that my YARN settings are higher than physical limits?


Which mapreduce 'pi' job settings can I use, to 'force' containers to 
use more than node physical resources?


Many thanks in advance!
Guido

Set yarn.nodemanager.resource.memory-mb higher than node physical memory

[. .]

Hi all,

I installed a basic 3 nodes Hadoop 2.9.1 cluster and playing with YARN
settings.
The 3 nodes has following configuration:
1 cpu / 1 core  / 512MB RAM

I wonder I was able to configure yarn-site.xml with following settings
(higher than node physical limits) and successfully run a mapreduce 'pi 1
10' job

quote...
  

yarn.resourcemanager.scheduler.classorg.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler



yarn.nodemanager.resource.memory-mb
5120
Amount of physical memory, in MB, that can be
allocated for containers. If set to -1 and
yarn.nodemanager.resource.detect-hardware-capabilities is true, it is
automatically calculated. In other cases, the default is
8192MB



yarn.nodemanager.resource.cpu-vcores
6
Number of CPU cores that can be allocated for
containers.

...unquote

Can anyone provide an explanation please?

Should 'yarn.nodemanager.vmem-check-enabled' and
'yarn.nodemanager.pmem-check-enabled' properties (set to 'true' as default)
check that my YARN settings are higher than physical limits?

Which mapreduce 'pi' job settings can I use, to 'force' containers to use
more than node physical resources?

Many thanks in advance!
Guido

Re: Hadoop HDFS Fault Injection

[Julien Laurenceau]

Hi,
Did you try anarchyape ?
Originally: https://github.com/david78k/anarchyape
my fork to avoid hard coded network interface as "eth0" :
https://github.com/julienlau/anarchyape

Regards,
JL

Le mer. 14 août 2019 à 15:12, Aleksander Buła <
ab370...@students.mimuw.edu.pl> a écrit :

> Hi,
>
> I would like to ask whether the Fault Injection Framework (
> https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/FaultInjectFramework.html)
> is still supported in the Hadoop HDFS?
>
> I think that this documentation was created around *v0.23* and was not
> updated since then. Additionally, I have done some repository digging and
> found out that the ant targets, mentioned in the documentation, were
> deleted in 2012. Right now none of the files in the repository defines
> these targets but there the projects contains multiple *.aj *files -
> therefore I assume they somehow can be used.
>
> Does anyone here know how to compile and run fault injection tests in a
> newer version of Hadoop (exactly* v2.6.0*)?  It would mean a world to me.
>
> Best regards
> Alex
>

Re: Hadoop HDFS Fault Injection

[Wei-Chiu Chuang]

Aleksander,
Yes I am aware of that doc but I've never seen any one maintaining that
piece of code in the last 4 years. And I don't think any one had ever used
that.

On Wed, Aug 14, 2019 at 5:12 AM Aleksander Buła <
ab370...@students.mimuw.edu.pl> wrote:

> Hi,
>
> I would like to ask whether the Fault Injection Framework (
> https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/FaultInjectFramework.html)
> is still supported in the Hadoop HDFS?
>
> I think that this documentation was created around *v0.23* and was not
> updated since then. Additionally, I have done some repository digging and
> found out that the ant targets, mentioned in the documentation, were
> deleted in 2012. Right now none of the files in the repository defines
> these targets but there the projects contains multiple *.aj *files -
> therefore I assume they somehow can be used.
>
> Does anyone here know how to compile and run fault injection tests in a
> newer version of Hadoop (exactly* v2.6.0*)?  It would mean a world to me.
>
> Best regards
> Alex
>

Hadoop HDFS Fault Injection

[Aleksander Buła]

Hi,

I would like to ask whether the Fault Injection Framework (
https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/FaultInjectFramework.html)
is still supported in the Hadoop HDFS?

I think that this documentation was created around *v0.23* and was not
updated since then. Additionally, I have done some repository digging and
found out that the ant targets, mentioned in the documentation, were
deleted in 2012. Right now none of the files in the repository defines
these targets but there the projects contains multiple *.aj *files -
therefore I assume they somehow can be used.

Does anyone here know how to compile and run fault injection tests in a
newer version of Hadoop (exactly* v2.6.0*)?  It would mean a world to me.

Best regards
Alex

Re: What are HDFS NFS “access times”?

[Matt Foley]

Hi Reed,
I think access time refers to the POSIX atime attribute for files, the “time of 
last access” as described here for instance [1].  While HDFS keeps a correct 
modification time (mtime), which is important, easy and cheap, it only keeps a 
very low-resolution sense of last access time, which is less important, and 
expensive to monitor and record, as described here [2] and here [3].  It 
doesn’t even expose this low-rez atime value in the `hadoop fs -stat` command; 
you need to use Java if you want to read it from HDFS apis.

However, to have a conforming NFS api, you must present atime, and so the HDFS 
NFS implementation does.  But first you have to configure it on.  The 
documentation says that the default value is 3,600,000 milliseconds (1 hour), 
but many sites have been advised to turn it off entirely by setting it to zero, 
to improve HDFS overall performance.  See for example here ( [4], section 
"Don’t let Reads become Writes”).  So if your site has turned off atime in 
HDFS, you will need to turn it back on to fully enable NFS.  Alternatively, you 
can maintain optimum efficiency by mounting NFS with the “noatime” option, as 
described in the document you reference.

I don’t know where the nfs3 daemon log file is, but it is almost certainly on 
the server node where you’ve configured the NFS service to be served from.  Log 
into it and check under /var/log, eg with `find /var/log -name ‘*nfs3*’ -print`

Hope this helps,
—Matt
—
Open Source Technologies @ Siri
`This is not a contribution`

[1] https://www.unixtutorial.org/atime-ctime-mtime-in-unix-filesystems
[2] https://issues.apache.org/jira/browse/HADOOP-1869
[3] 
https://superuser.com/questions/464290/why-is-cat-not-changing-the-access-time
[4] 
https://community.hortonworks.com/articles/43861/scaling-the-hdfs-namenode-part-4-avoiding-performa.html


On Aug 13, 2019, at 6:43 PM, Reed Villanueva  wrote:

Having a problem with HDFS NFS, addressed on another site 

 where it is recommended to set hdfs-site.xml like...

  
dfs.namenode.accesstime.precision  
360  

The access time for HDFS file is precise upto this value. The default value is 
1 hour. Setting a value of 0 disables access times for HDFS.  
 

Am confused about what exactly "access times for HDFS" means / is. Looking at 
the hadoop docs 
,
 was still not able to determine. Could someone give better understanding as to 
what this is doing? Also, where is the nfs3 daemon log file?