[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. IMPALA-8785: give debug docker images a different name * Build scripts are generalised to have different targets for release and debug images. * Added new targets for the debug images: docker_debug_images, statestored_debug images. The release images still have the same names. * Separate build contexts are set up for the different base images. * The debug or release base image can be specified as the FROM for the daemon images. * start-impala-cluster.py picks the correct images for the build type Future work: We would like to generalise this to allow building from non-ubuntu-16.04 base images. This probably requires another layer of dockerfiles to specify a base image for impala_base with the required packages installed. Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Reviewed-on: http://gerrit.cloudera.org:8080/13905 Reviewed-by: Joe McDonnell Tested-by: Impala Public Jenkins --- M bin/jenkins/dockerized-impala-run-tests.sh M bin/start-impala-cluster.py M docker/CMakeLists.txt M docker/catalogd/Dockerfile M docker/impalad_coord_exec/Dockerfile M docker/impalad_coordinator/Dockerfile M docker/impalad_executor/Dockerfile M docker/setup_build_context.py M docker/statestored/Dockerfile 9 files changed, 96 insertions(+), 45 deletions(-) Approvals: Joe McDonnell: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 12 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 11: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 11 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 30 Jul 2019 23:36:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4081/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 10 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 30 Jul 2019 17:10:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 11: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4691/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 11 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 30 Jul 2019 17:04:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 11: Code-Review+2 This looks good to me -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 11 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 30 Jul 2019 16:37:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 9: (3 comments) http://gerrit.cloudera.org:8080/#/c/13905/9/.gitignore File .gitignore: http://gerrit.cloudera.org:8080/#/c/13905/9/.gitignore@47 PS9, Line 47: docker/catalogd/Dockerfile : docker/statestored/Dockerfile : docker/impalad*/Dockerfile > I think we need to remove these Done http://gerrit.cloudera.org:8080/#/c/13905/9/docker/CMakeLists.txt File docker/CMakeLists.txt: http://gerrit.cloudera.org:8080/#/c/13905/9/docker/CMakeLists.txt@22 PS9, Line 22: target build > Nit: "target to build" Done http://gerrit.cloudera.org:8080/#/c/13905/9/docker/CMakeLists.txt@63 PS9, Line 63: Docker does not support : # variables in the FROM clause, so some kind of templating is needed. Using sed feels : # like a hack, but is an accepted practice for generating Dockerfiles. > This comment is stale now that we are using build args. Done -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 9 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 30 Jul 2019 16:29:44 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Hello Anurag Mantripragada, Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13905 to look at the new patch set (#10). Change subject: IMPALA-8785: give debug docker images a different name .. IMPALA-8785: give debug docker images a different name * Build scripts are generalised to have different targets for release and debug images. * Added new targets for the debug images: docker_debug_images, statestored_debug images. The release images still have the same names. * Separate build contexts are set up for the different base images. * The debug or release base image can be specified as the FROM for the daemon images. * start-impala-cluster.py picks the correct images for the build type Future work: We would like to generalise this to allow building from non-ubuntu-16.04 base images. This probably requires another layer of dockerfiles to specify a base image for impala_base with the required packages installed. Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 --- M bin/jenkins/dockerized-impala-run-tests.sh M bin/start-impala-cluster.py M docker/CMakeLists.txt M docker/catalogd/Dockerfile M docker/impalad_coord_exec/Dockerfile M docker/impalad_coordinator/Dockerfile M docker/impalad_executor/Dockerfile M docker/setup_build_context.py M docker/statestored/Dockerfile 9 files changed, 96 insertions(+), 45 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/13905/10 -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 10 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 9: (3 comments) This makes sense. A couple small comments http://gerrit.cloudera.org:8080/#/c/13905/9/.gitignore File .gitignore: http://gerrit.cloudera.org:8080/#/c/13905/9/.gitignore@47 PS9, Line 47: docker/catalogd/Dockerfile : docker/statestored/Dockerfile : docker/impalad*/Dockerfile I think we need to remove these http://gerrit.cloudera.org:8080/#/c/13905/9/docker/CMakeLists.txt File docker/CMakeLists.txt: http://gerrit.cloudera.org:8080/#/c/13905/9/docker/CMakeLists.txt@22 PS9, Line 22: target build Nit: "target to build" http://gerrit.cloudera.org:8080/#/c/13905/9/docker/CMakeLists.txt@63 PS9, Line 63: Docker does not support : # variables in the FROM clause, so some kind of templating is needed. Using sed feels : # like a hack, but is an accepted practice for generating Dockerfiles. This comment is stale now that we are using build args. -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 9 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 30 Jul 2019 14:31:12 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 9: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 9 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 30 Jul 2019 07:23:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 8: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4685/ -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 30 Jul 2019 00:49:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 9: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4686/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 9 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 30 Jul 2019 00:49:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 8: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4685/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 30 Jul 2019 00:49:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4044/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 27 Jul 2019 00:43:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 8: Rebased onto Lars' change -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 27 Jul 2019 00:01:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Hello Anurag Mantripragada, Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13905 to look at the new patch set (#8). Change subject: IMPALA-8785: give debug docker images a different name .. IMPALA-8785: give debug docker images a different name * Build scripts are generalised to have different targets for release and debug images. * Added new targets for the debug images: docker_debug_images, statestored_debug images. The release images still have the same names. * Separate build contexts are set up for the different base images. * The debug or release base image can be specified as the FROM for the daemon images. * start-impala-cluster.py picks the correct images for the build type Future work: We would like to generalise this to allow building from non-ubuntu-16.04 base images. This probably requires another layer of dockerfiles to specify a base image for impala_base with the required packages installed. Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 --- M .gitignore M bin/jenkins/dockerized-impala-run-tests.sh M bin/start-impala-cluster.py M docker/CMakeLists.txt M docker/catalogd/Dockerfile M docker/impalad_coord_exec/Dockerfile M docker/impalad_coordinator/Dockerfile M docker/impalad_executor/Dockerfile M docker/setup_build_context.py M docker/statestored/Dockerfile 10 files changed, 101 insertions(+), 45 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/13905/8 -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 7: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Jul 2019 12:57:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3993/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Jul 2019 07:03:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4656/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Jul 2019 06:23:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Hello Anurag Mantripragada, Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13905 to look at the new patch set (#7). Change subject: IMPALA-8785: give debug docker images a different name .. IMPALA-8785: give debug docker images a different name * Build scripts are generalised to have different targets for release and debug images. * Added new targets for the debug images: docker_debug_images, statestored_debug images. The release images still have the same names. * Separate build contexts are set up for the different base images. * The debug or release base image can be specified as the FROM for the daemon images. * start-impala-cluster.py picks the correct images for the build type Future work: We would like to generalise this to allow building from non-ubuntu-16.04 base images. This probably requires another layer of dockerfiles to specify a base image for impala_base with the required packages installed. Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 --- M .gitignore M bin/jenkins/dockerized-impala-run-tests.sh M bin/start-impala-cluster.py M docker/CMakeLists.txt M docker/catalogd/Dockerfile M docker/impalad_coord_exec/Dockerfile M docker/impalad_coordinator/Dockerfile M docker/impalad_executor/Dockerfile M docker/setup_build_context.py M docker/statestored/Dockerfile 10 files changed, 100 insertions(+), 44 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/13905/7 -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 6: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4648/ -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Jul 2019 05:03:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3985/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 24 Jul 2019 23:15:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/13905/6/.gitignore File .gitignore: PS6: Need to undo these changes -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 24 Jul 2019 22:36:39 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 6: I was able to make it work! -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 24 Jul 2019 22:35:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4648/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 24 Jul 2019 22:36:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Hello Anurag Mantripragada, Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13905 to look at the new patch set (#6). Change subject: IMPALA-8785: give debug docker images a different name .. IMPALA-8785: give debug docker images a different name * Build scripts are generalised to have different targets for release and debug images. * Added new targets for the debug images: docker_debug_images, statestored_debug images. The release images still have the same names. * Separate build contexts are set up for the different base images. * The debug or release base image can be specified as the FROM for the daemon images. * start-impala-cluster.py picks the correct images for the build type Future work: We would like to generalise this to allow building from non-ubuntu-16.04 base images. This probably requires another layer of dockerfiles to specify a base image for impala_base with the required packages installed. Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 --- M .gitignore M bin/jenkins/dockerized-impala-run-tests.sh M bin/start-impala-cluster.py M docker/CMakeLists.txt M docker/catalogd/Dockerfile M docker/impalad_coord_exec/Dockerfile M docker/impalad_coordinator/Dockerfile M docker/impalad_executor/Dockerfile M docker/setup_build_context.py M docker/statestored/Dockerfile 10 files changed, 100 insertions(+), 44 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/13905/6 -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 5: For IMPALA-8770 I'm leaning towards just using different base dockerfiles for different distros. Using template substitution to install arbitrary packages or do whatever other customisation seems like the wrong mechanism. So I'll go with --build-arg -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 24 Jul 2019 18:52:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/13905/5/docker/CMakeLists.txt File docker/CMakeLists.txt: http://gerrit.cloudera.org:8080/#/c/13905/5/docker/CMakeLists.txt@62 PS5, Line 62: Docker does not support : # variables in the FROM clause > It does work for me now, wonder what I did wrong first time. Before I go ba The build args would handle the build_type stuff in this change. I think it could potentially handle IMPALA-8770. I would maybe lean towards using the build args for now. Even so, I don't think I have strong preferences one way or the other. The templating isn't that bad in terms of complexity. -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 24 Jul 2019 18:07:29 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/13905/5/docker/CMakeLists.txt File docker/CMakeLists.txt: http://gerrit.cloudera.org:8080/#/c/13905/5/docker/CMakeLists.txt@62 PS5, Line 62: Docker does not support : # variables in the FROM clause > I tried to do this but didn't get it to work. Maybe I just made a mistake a It does work for me now, wonder what I did wrong first time. Before I go back and undo the templating, you think that the --build-arg mechanism is the right way to go? I think I agree... * Pros for templating: it's more extensible in future, matches standard practices in the docker community. * Cons for templating: it adds a little more complexity that we don't need right now -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 24 Jul 2019 17:54:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/13905/5/docker/CMakeLists.txt File docker/CMakeLists.txt: http://gerrit.cloudera.org:8080/#/c/13905/5/docker/CMakeLists.txt@62 PS5, Line 62: Docker does not support : # variables in the FROM clause > When I was looking into this, it seemed like ARG might work: I tried to do this but didn't get it to work. Maybe I just made a mistake and gave up too soon. I did see some docs that suggested it only supported substituting the version field, but it's entirely possible I just misunderstood. I'll have another go. -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 24 Jul 2019 17:47:32 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13905 ) Change subject: IMPALA-8785: give debug docker images a different name .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3977/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 24 Jul 2019 17:38:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Joe McDonnell has posted comments on this change. (
http://gerrit.cloudera.org:8080/13905 )
Change subject: IMPALA-8785: give debug docker images a different name
..
Patch Set 5:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13905/5/docker/CMakeLists.txt
File docker/CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/13905/5/docker/CMakeLists.txt@62
PS5, Line 62: Docker does not support
: # variables in the FROM clause
When I was looking into this, it seemed like ARG might work:
https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact
I did a quick test where I changed impalad_coordinator so that it looks like
this:
ARG BASE_IMAGE=impala_base
FROM ${BASE_IMAGE}
and it seems to work. Then you should be able to pass in an argument to docker
build with --build-arg =
--
To view, visit http://gerrit.cloudera.org:8080/13905
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244
Gerrit-Change-Number: 13905
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Anurag Mantripragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Wed, 24 Jul 2019 17:35:31 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8785: give debug docker images a different name
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13905 Change subject: IMPALA-8785: give debug docker images a different name .. IMPALA-8785: give debug docker images a different name * Build scripts are generalised to have different targets for release and debug images. * Added new targets for the debug images: docker_debug_images, statestored_debug images. The release images still have the same names. * Separate build contexts are set up for the different base images. * We need a mechanism to substitute the base image into the Dockerfiles. Sed is used, consistent with https://github.com/docker-library. * start-impala-cluster.py picks the correct images for the build type Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 --- M .gitignore M bin/jenkins/dockerized-impala-run-tests.sh M bin/start-impala-cluster.py M docker/CMakeLists.txt R docker/catalogd/Dockerfile.template R docker/impalad_coord_exec/Dockerfile.template R docker/impalad_coordinator/Dockerfile.template R docker/impalad_executor/Dockerfile.template M docker/setup_build_context.py R docker/statestored/Dockerfile.template 10 files changed, 96 insertions(+), 44 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/13905/5 -- To view, visit http://gerrit.cloudera.org:8080/13905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244 Gerrit-Change-Number: 13905 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
