TLDR;
I proposed to create a separated git repository for ozone docker images
in HDDS-851 (hadoop-docker-ozone.git)
If there is no objections in the next 3 days I will ask an Apache Member
to create the repository.
LONG VERSION:
In HADOOP-14898 multiple docker containers and helper scripts are
created for Hadoop.
The main goal was to:
1.) help the development with easy-to-use docker images
2.) provide official hadoop images to make it easy to test new features
As of now we have:
- apache/hadoop-runner image (which contains the required dependency
but no hadoop)
- apache/hadoop:2 and apache/hadoop:3 images (to try out latest hadoop
from 2/3 lines)
The base image to run hadoop (apache/hadoop-runner) is also heavily used
for Ozone distribution/development.
The Ozone distribution contains docker-compose based cluster definitions
to start various type of clusters and scripts to do smoketesting. (See
HADOOP-16063 for more details).
Note: I personally believe that these definitions help a lot to start
different type of clusters. For example it could be tricky to try out
router based federation as it requires multiple HA clusters. But with a
simple docker-compose definition [1] it could be started under 3
minutes. (HADOOP-16063 is about creating these definitions for various
hdfs/yarn use cases)
As of now we have dedicated branches in the hadoop git repository for
the docker images (docker-hadoop-runner, docker-hadoop-2,
docker-hadoop-3). It turns out that a separated repository would be more
effective as the dockerhub can use only full branch names as tags.
We would like to provide ozone docker images to make the evaluation as
easy as 'docker run -d apache/hadoop-ozone:0.3.0', therefore in HDDS-851
we agreed to create a separated repository for the hadoop-ozone docker
images.
If this approach works well we can also move out the existing
docker-hadoop-2/docker-hadoop-3/docker-hadoop-runner branches from
hadoop.git to an other separated hadoop-docker.git repository)
Please let me know if you have any comments,
Thanks,
Marton
1: see
https://github.com/flokkr/runtime-compose/tree/master/hdfs/routerfeder
as an example
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
