+1 to add Hadoop docker image for easier testing / prototyping, it gonna be super helpful!
Thanks, Wangda On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi < miklos.szeg...@cloudera.com> wrote: > Marton, thank you for working on this. I think Official Docker images for > Hadoop would be very useful for a lot of reasons. I think that it is better > to have a coordinated effort with production ready base images with > dependent images for prototyping. Does anyone else have an opinion about > this? > > Thank you, > Miklos > > On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <h...@anzix.net> wrote: > > > > > TL;DR: I propose to create official hadoop images and upload them to the > > dockerhub. > > > > GOAL/SCOPE: I would like improve the existing documentation with > > easy-to-use docker based recipes to start hadoop clusters with various > > configuration. > > > > The images also could be used to test experimental features. For example > > ozone could be tested easily with these compose file and configuration: > > > > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6 > > > > Or even the configuration could be included in the compose file: > > > > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc > > ker-compose.yaml > > > > I would like to create separated example compose files for federation, > ha, > > metrics usage, etc. to make it easier to try out and understand the > > features. > > > > CONTEXT: There is an existing Jira https://issues.apache.org/jira > > /browse/HADOOP-13397 > > But it’s about a tool to generate production quality docker images > > (multiple types, in a flexible way). If no objections, I will create a > > separated issue to create simplified docker images for rapid prototyping > > and investigating new features. And register the branch to the dockerhub > to > > create the images automatically. > > > > MY BACKGROUND: I am working with docker based hadoop/spark clusters quite > > a while and run them succesfully in different environments (kubernetes, > > docker-swarm, nomad-based scheduling, etc.) My work is available from > here: > > https://github.com/flokkr but they could handle more complex use cases > > (eg. instrumenting java processes with btrace, or read/reload > configuration > > from consul). > > And IMHO in the official hadoop documentation it’s better to suggest to > > use official apache docker images and not external ones (which could be > > changed). > > > > Please let me know if you have any comments. > > > > Marton > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > > > >
