It would be very helpful for testing the RC. To vote a RC, committers and PMCs usually spend lots of time to compile, deploy the RC, do several sanity tests, then +1 for the RC. The docker image potentially saves the compilation and deployment time, and people can do more tests.
Best, Yufei On Wed, Sep 13, 2017 at 11:19 AM, Wangda Tan <wheele...@gmail.com> wrote: > +1 to add Hadoop docker image for easier testing / prototyping, it gonna be > super helpful! > > Thanks, > Wangda > > On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi < > miklos.szeg...@cloudera.com> wrote: > > > Marton, thank you for working on this. I think Official Docker images for > > Hadoop would be very useful for a lot of reasons. I think that it is > better > > to have a coordinated effort with production ready base images with > > dependent images for prototyping. Does anyone else have an opinion about > > this? > > > > Thank you, > > Miklos > > > > On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <h...@anzix.net> wrote: > > > > > > > > TL;DR: I propose to create official hadoop images and upload them to > the > > > dockerhub. > > > > > > GOAL/SCOPE: I would like improve the existing documentation with > > > easy-to-use docker based recipes to start hadoop clusters with various > > > configuration. > > > > > > The images also could be used to test experimental features. For > example > > > ozone could be tested easily with these compose file and configuration: > > > > > > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6 > > > > > > Or even the configuration could be included in the compose file: > > > > > > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc > > > ker-compose.yaml > > > > > > I would like to create separated example compose files for federation, > > ha, > > > metrics usage, etc. to make it easier to try out and understand the > > > features. > > > > > > CONTEXT: There is an existing Jira https://issues.apache.org/jira > > > /browse/HADOOP-13397 > > > But it’s about a tool to generate production quality docker images > > > (multiple types, in a flexible way). If no objections, I will create a > > > separated issue to create simplified docker images for rapid > prototyping > > > and investigating new features. And register the branch to the > dockerhub > > to > > > create the images automatically. > > > > > > MY BACKGROUND: I am working with docker based hadoop/spark clusters > quite > > > a while and run them succesfully in different environments (kubernetes, > > > docker-swarm, nomad-based scheduling, etc.) My work is available from > > here: > > > https://github.com/flokkr but they could handle more complex use cases > > > (eg. instrumenting java processes with btrace, or read/reload > > configuration > > > from consul). > > > And IMHO in the official hadoop documentation it’s better to suggest > to > > > use official apache docker images and not external ones (which could be > > > changed). > > > > > > Please let me know if you have any comments. > > > > > > Marton > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > > > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > > > > > > > >
