> It would be very helpful for testing the RC. For testing and voting, I have been using docker containers for a while, see code at: https://github.com/weiqingy/caochong <https://github.com/weiqingy/caochong>
> TL;DR: I propose to create official hadoop images and upload them to the > dockerhub I’m +1 on this idea. The “official” docker image basically means a commitment to maintain well documented and broadly tested images, which seems not a burden to us. Ceph has a community docker project https://github.com/ceph/ceph-docker <https://github.com/ceph/ceph-docker> and I think our scope here is similar to it. Mingliang > On Sep 13, 2017, at 11:39 AM, Yufei Gu <flyrain...@gmail.com> wrote: > > It would be very helpful for testing the RC. To vote a RC, committers and > PMCs usually spend lots of time to compile, deploy the RC, do several > sanity tests, then +1 for the RC. The docker image potentially saves the > compilation and deployment time, and people can do more tests. > > Best, > > Yufei > > On Wed, Sep 13, 2017 at 11:19 AM, Wangda Tan <wheele...@gmail.com> wrote: > >> +1 to add Hadoop docker image for easier testing / prototyping, it gonna be >> super helpful! >> >> Thanks, >> Wangda >> >> On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi < >> miklos.szeg...@cloudera.com> wrote: >> >>> Marton, thank you for working on this. I think Official Docker images for >>> Hadoop would be very useful for a lot of reasons. I think that it is >> better >>> to have a coordinated effort with production ready base images with >>> dependent images for prototyping. Does anyone else have an opinion about >>> this? >>> >>> Thank you, >>> Miklos >>> >>> On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <h...@anzix.net> wrote: >>> >>>> >>>> TL;DR: I propose to create official hadoop images and upload them to >> the >>>> dockerhub. >>>> >>>> GOAL/SCOPE: I would like improve the existing documentation with >>>> easy-to-use docker based recipes to start hadoop clusters with various >>>> configuration. >>>> >>>> The images also could be used to test experimental features. For >> example >>>> ozone could be tested easily with these compose file and configuration: >>>> >>>> https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6 >>>> >>>> Or even the configuration could be included in the compose file: >>>> >>>> https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc >>>> ker-compose.yaml >>>> >>>> I would like to create separated example compose files for federation, >>> ha, >>>> metrics usage, etc. to make it easier to try out and understand the >>>> features. >>>> >>>> CONTEXT: There is an existing Jira https://issues.apache.org/jira >>>> /browse/HADOOP-13397 >>>> But it’s about a tool to generate production quality docker images >>>> (multiple types, in a flexible way). If no objections, I will create a >>>> separated issue to create simplified docker images for rapid >> prototyping >>>> and investigating new features. And register the branch to the >> dockerhub >>> to >>>> create the images automatically. >>>> >>>> MY BACKGROUND: I am working with docker based hadoop/spark clusters >> quite >>>> a while and run them succesfully in different environments (kubernetes, >>>> docker-swarm, nomad-based scheduling, etc.) My work is available from >>> here: >>>> https://github.com/flokkr but they could handle more complex use cases >>>> (eg. instrumenting java processes with btrace, or read/reload >>> configuration >>>> from consul). >>>> And IMHO in the official hadoop documentation it’s better to suggest >> to >>>> use official apache docker images and not external ones (which could be >>>> changed). >>>> >>>> Please let me know if you have any comments. >>>> >>>> Marton >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org >>>> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org >>>> >>>> >>> >>
