Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs ...................................................................... Patch Set 3: (2 comments) Went through the logic; found a preexisting failure case not even described in the original ticket. http://gerrit.cloudera.org:8080/#/c/15898/3/docker/entrypoint.sh File docker/entrypoint.sh: http://gerrit.cloudera.org:8080/#/c/15898/3/docker/entrypoint.sh@319 PS3, Line 319: copy_cluster_logs I'd recommend moving this line before L317: according to its comments, that line attempts to remove the very symlinks that copy_cluster_logs" is trying to save for later (although the exact flag and predicate combination seems to be wrong there). http://gerrit.cloudera.org:8080/#/c/15898/3/docker/entrypoint.sh@319 PS3, Line 319: copy_cluster_logs There is also another problem here (not covered in the original ticket): For each instantiation of the container (whether for build or test purposes) the driver script test_with_docker.py mounts a different log directory at the containers "/log" mount point (see L580-581: https://github.com/apache/impala/blob/master/docker/test-with-docker.py#L580, L655, L815 https://github.com/apache/impala/blob/master/docker/test-with-docker.py#L815). This means anything created there from within the container will not be present in the log tree of a different container run, unless there is a mechanism in test-with-docker.py that replicates or recreates the artifact. Add to this that the cluster log symlinks are created by testdata/cluster/admin in the function create_cluster(), which is run only in the build container, but not in any of the test containers. This has the end result that none of the test containers will actually contain the symlinks from /logs/cluster to the node logs, because the test containers run only testdata/cluster/admin start_cluster(), which does not ensure that the symlinks are present. I see two ways out of this. a) either fix testdata/cluster/admin so that start_cluster() (re)creates these symlink if they are gone b) or fix entrypoint.sh to the same effect. Once this is fixed, then the call to copy_cluster_logs should also be invoked from boot_container() (or possibly from test_suite() ) to ensure that the symlinks are there to capture the log files, so that clpy_cluster_logs can later pick them up and archive them. -- To view, visit http://gerrit.cloudera.org:8080/15898 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e Gerrit-Change-Number: 15898 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Garaguly <zgarag...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Laszlo Gaal <laszlo.g...@cloudera.com> Gerrit-Reviewer: Zoltan Garaguly <zgarag...@cloudera.com> Gerrit-Comment-Date: Mon, 18 May 2020 17:40:36 +0000 Gerrit-HasComments: Yes
