[
https://issues.apache.org/jira/browse/YARN-9229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752563#comment-16752563
]
Eric Yang edited comment on YARN-9229 at 1/25/19 7:19 PM:
----------------------------------------------------------
Environment: 5 Nodes VM cluster.
Model name: Intel Xeon E312xx (Sandy Bridge)
Cpu MHz: 2399.996
Network: 1GB Ethernet
Software: CentOS 7.3, Hadoop 3.3.0-SNAPSHOT and ZooKeeper 3.4.5, Kerberos
Enabled cluster.
Configure Docker /etc/docker/daemon.json to include:
{code}
{
"insecure-registries" : [ "registry.docker-reg.hbase.ycluster:5000" ]
}
{code}
Where registry.docker-reg.hbase.ycluster is the generated hostname that will be
started by YARN service to run Docker registry.
Standard Hadoop configuration with NFS configuration:
{code}
<property>
<name>nfs.exports.allowed.hosts</name>
<value>* rw</value>
</property>
<property>
<name>nfs.file.dump.dir</name>
<value>/tmp/.hdfs-nfs</value>
</property>
<property>
<name>nfs.kerberos.principal</name>
<value>nfs/_h...@example.com</value>
</property>
<property>
<name>nfs.keytab.file</name>
<value>/etc/security/keytabs/nfs.service.keytab</value>
</property>
{code}
Run NFS Gateway on all datanodes as hdfs user using: {code}$
$HADOOP_HOME/bin/hdfs nfs3{code}
On each datanode, nfs mount point is exposed to /hdfs, using:
{code}# mount -t nfs -o vers=3,proto=tcp,nolock,noacl,sync $DN_IP:/ /hdfs{code}
Docker Registry is started using YARN service:
registry.json
{code}{
"name": "docker-registry",
"version": "1.0",
"kerberos_principal" : {
"principal_name" : "hbase/_h...@example.com",
"keytab" : "file:///etc/security/keytabs/hbase.service.keytab"
},
"components" :
[
{
"name": "registry",
"number_of_containers": 1,
"artifact": {
"id": "registry:latest",
"type": "DOCKER"
},
"resource": {
"cpus": 1,
"memory": "256"
},
"run_privileged_container": true,
"configuration": {
"env": {
"YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE":"true",
"YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS":"/hdfs/apps/docker/registry:/var/lib/registry"
},
"properties": {
"docker.network": "host"
}
}
}
]
}
{code}
YARN service configures docker mounts from /hdfs/apps/docker/registry to
/var/lib/registry inside docker container.
{code}yarn app -launch docker-reg /tmp/registry.json{code}
was (Author: eyang):
Environment: 5 Nodes VM cluster.
Model name: Intel Xeon E312xx (Sandy Bridge)
Cpu MHz: 2399.996
Network: 1GB Ethernet
Software: CentOS 7.3, Hadoop 3.3.0-SNAPSHOT and ZooKeeper 3.4.5, Kerberos
Enabled cluster.
Configure Docker /etc/docker/daemon.json to include:
{code}
{
"insecure-registries" : [ "registry.docker-reg.hbase.ycluster:5000" ]
}
{code}
Where registry.docker-reg.hbase.ycluster is the generated hostname that will be
started by YARN service to run Docker registry.
Standard Hadoop configuration with NFS configuration:
{code}
<property>
<name>nfs.exports.allowed.hosts</name>
<value>* rw</value>
</property>
<property>
<name>nfs.file.dump.dir</name>
<value>/tmp/.hdfs-nfs</value>
</property>
<property>
<name>nfs.kerberos.principal</name>
<value>nfs/_h...@example.com</value>
</property>
<property>
<name>nfs.keytab.file</name>
<value>/etc/security/keytabs/nfs.service.keytab</value>
</property>
{code}
Run NFS Gateway on all datanodes as hdfs user using: {code}$
$HADOOP_HOME/bin/hdfs nfs3{code}
On each datanode, nfs mount point is exposed to /hdfs, using:
{code}# mount -t nfs -o vers=3,proto=tcp,nolock,noacl,sync $DN_IP:/ /hdfs{code}
Docker Registry is started using YARN service:
registry.json
{code}{
"name": "docker-registry",
"version": "1.0",
"kerberos_principal" : {
"principal_name" : "hbase/_h...@example.com",
"keytab" : "file:///etc/security/keytabs/hbase.service.keytab"
},
"components" :
[
{
"name": "registry",
"number_of_containers": 1,
"artifact": {
"id": "registry:latest",
"type": "DOCKER"
},
"resource": {
"cpus": 1,
"memory": "256"
},
"configuration": {
"env": {
"YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE":"true",
"YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS":"/hdfs/apps/docker/registry:/var/lib/registry"
},
"properties": {
"docker.network": "host"
}
}
}
]
}
{code}
YARN service configures docker mounts from /hdfs/apps/docker/registry to
/var/lib/registry inside docker container.
{code}yarn app -launch docker-reg /tmp/registry.json{code}
> Document docker registry deployment with NFS Gateway
> ----------------------------------------------------
>
> Key: YARN-9229
> URL: https://issues.apache.org/jira/browse/YARN-9229
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Major
>
> The goal of this task is to demonstrate running docker pull of image from
> HDFS via NFS gateway. Document the pros and cons for using NFS gateway.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
