Re: [DISCUSS] Docker build process

2019-05-06 Thread Elek, Marton
Thanks the answers Eric Yang, I think we have similar view about how the releases are working and what you wrote is exactly the reason why I prefer the current method (docker image creation from separated branch) instead of the proposed one (create images from maven). 1. Not all the branches can b

Re: [DISCUSS] Docker build process

2019-05-04 Thread Eric Yang
See comments inline. On Fri, Mar 22, 2019 at 2:06 AM Elek, Marton wrote: > > > Thanks the answer, > > I agree, sha256 based tags seems to be more safe and bumping versions > only after some tests. > > > Let's say we have multiple hadoop docker images: > > apache/hadoop:3.2.0 > apache/hadoop:3.1.

Re: [DISCUSS] Docker build process

2019-03-22 Thread Elek, Marton
Thanks the answer, I agree, sha256 based tags seems to be more safe and bumping versions only after some tests. Let's say we have multiple hadoop docker images: apache/hadoop:3.2.0 apache/hadoop:3.1.2 apache/hadoop:2.9.2 apache/hadoop:2.8.5 apache/hadoop:2.7.7 If I understood well, your pr

Re: [DISCUSS] Docker build process

2019-03-21 Thread Eric Yang
The flexibility of date appended release number is equivalent to maven snapshot or Docker latest image convention, machine can apply timestamp better than human. By using the Jenkins release process, this can be done with little effort. For official release, it is best to use Docker image dige

Re: [DISCUSS] Docker build process

2019-03-21 Thread Elek, Marton
> If versioning is done correctly, older branches can have the same docker > subproject, and Hadoop 2.7.8 can be released for older Hadoop branches. We > don't generate timeline paradox to allow changing the history of Hadoop > 2.7.1. That release has passed and let it stay that way. I und

Re: [DISCUSS] Docker build process

2019-03-19 Thread Eric Yang
, Eric From: Jonathan Eagles Date: Tuesday, March 19, 2019 at 11:48 AM To: Eric Yang Cc: "Elek, Marton" , Hadoop Common , "yarn-...@hadoop.apache.org" , Hdfs-dev , Eric Badger , Eric Payne , Jim Brennan Subject: Re: [DISCUSS] Docker build process This email discussion thr

Re: [DISCUSS] Docker build process

2019-03-19 Thread Eric Yang
Hi Arpit, On Docker Hub, Hadoop tagged with version number that looks like: docker-hadoop-runner-latest, or jdk11. It is hard to tell if jdk11 image is Hadoop 3 or Hadoop 2 because there is no consistency in tag format usage. This is my reasoning against tag as your heart desired because flex

Re: [DISCUSS] Docker build process

2019-03-19 Thread Arpit Agarwal
Hi Eric, > Dockerfile is most likely to change to apply the security fix. I am not sure this is always. Marton’s point about revising docker images independent of Hadoop versions is valid. > When maven release is automated through Jenkins, this is a breeze > of clicking a button. Jenkins eve

Re: [DISCUSS] Docker build process

2019-03-19 Thread Jonathan Eagles
This email discussion thread is the result of failing to reach consensus in the JIRA. If you participate in this discussion thread, please recognize that a considerable effort has been made by contributors on this JIRA. On the other hand, contributors to this JIRA need to listen carefully to the co

Re: [DISCUSS] Docker build process

2019-03-19 Thread Jim Brennan
I agree with Steve and Marton. I am ok with having the docker build as an option, but I don't want it to be the default. Jim On Tue, Mar 19, 2019 at 12:19 PM Eric Yang wrote: > Hi Marton, > > Thank you for your input. I agree with most of what you said with a few > exceptions. Security fix

Re: [DISCUSS] Docker build process

2019-03-19 Thread Eric Yang
Hi Marton, Thank you for your input. I agree with most of what you said with a few exceptions. Security fix should result in a different version of the image instead of replace of a certain version. Dockerfile is most likely to change to apply the security fix. If it did not change, the sou

Re: [DISCUSS] Docker build process

2019-03-19 Thread Elek, Marton
Thank you Eric to describe the problem. I have multiple small comments, trying to separate them. I. separated vs in-build container image creation > The disadvantages are: > > 1. Require developer to have access to docker. > 2. Default build takes longer. These are not the only disadvanta

Re: [DISCUSS] Docker build process

2019-03-18 Thread Eric Yang
Loughran Date: Monday, March 18, 2019 at 3:36 AM To: Eric Yang Cc: Hadoop Common , "yarn-...@hadoop.apache.org" , Hdfs-dev , Eric Badger , Eric Payne , Jonathan Eagles , Jim Brennan , "Elek, Marton" Subject: Re: [DISCUSS] Docker build process I'm not enthusiastic abou

Re: [DISCUSS] Docker build process

2019-03-18 Thread Steve Loughran
I'm not enthusiastic about making the docker build process mandatory. It's bad enough having to remember to type -DskipShade to avoid a 5-10 minute delay every time I do a build of a different branch, which I have to do every single time I change from one PR to another. I do not see why the docker

[DISCUSS] Docker build process

2019-03-13 Thread Eric Yang
Hi Hadoop developers, In the recent months, there were various discussions on creating docker build process for Hadoop. There was convergence to make docker build process inline in the mailing list last month when Ozone team is planning new repository for Hadoop/ozone docker images. New featu