[jira] [Commented] (IMPALA-8392) make -j $IMPALA_BUILD_THREADS docker_images fails with OSError: [Errno 17] File exists: '/home/ubuntu/Impala/docker/build_context'

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-8392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820545#comment-16820545
 ] 

ASF subversion and git services commented on IMPALA-8392:
-

Commit c8dbad8ed989258c241affb40a059fc083f9a90a in impala's branch 
refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=c8dbad8 ]

IMPALA-8392: fix parallel docker_images build

I made the other targets depend on targets, not the
timestamp file, according to the suggested solution
in:
https://gitlab.kitware.com/cmake/cmake/issues/17585

Testing:
Ran "make -j 8 docker_images" locally, which now succeeds.

Running dockerised tests.

Change-Id: Idb658ee156eb9b186ff3fcc3e4a40ad87ed7c0ce
Reviewed-on: http://gerrit.cloudera.org:8080/13053
Reviewed-by: Impala Public Jenkins 
Tested-by: Impala Public Jenkins 


> make -j $IMPALA_BUILD_THREADS docker_images fails with OSError: [Errno 17] 
> File exists: '/home/ubuntu/Impala/docker/build_context'
> --
>
> Key: IMPALA-8392
> URL: https://issues.apache.org/jira/browse/IMPALA-8392
> Project: IMPALA
>  Issue Type: Sub-task
>  Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Minor
>
> Looks like there is a bug in the CMake file. I didn't see this with serial 
> make or with ninja at all. I use ninja locally which is why I didn't notice 
> this while developing it.
> {noformat}
> 02:45:52 Scanning dependencies of target catalogd_image
> 02:45:52 Scanning dependencies of target coord_exec_image
> 02:45:52 Scanning dependencies of target statestored_image
> 02:45:52 Scanning dependencies of target coordinator_image
> 02:45:52 Scanning dependencies of target executor_image
> 02:45:52 [ 97%] Creating impala base build context.
> 02:45:52 [ 97%] Creating impala base build context.
> 02:45:52 [ 97%] Creating impala base build context.
> 02:45:52 [ 98%] Creating impala base build context.
> 02:45:52 [ 98%] Creating impala base build context.
> 02:45:52 Traceback (most recent call last):
> 02:45:52 Traceback (most recent call last):
> 02:45:52 Traceback (most recent call last):
> 02:45:52   File "./setup_build_context.py", line 38, in 
> 02:45:52   File "./setup_build_context.py", line 38, in 
> 02:45:52   File "./setup_build_context.py", line 38, in 
> 02:45:52 os.makedirs(OUTPUT_DIR)
> 02:45:52 os.makedirs(OUTPUT_DIR)
> 02:45:52 os.makedirs(OUTPUT_DIR)
> 02:45:52   File 
> "/home/ubuntu/Impala/bin/../infra/python/env/lib/python2.7/os.py", line 157, 
> in makedirs
> 02:45:52   File 
> "/home/ubuntu/Impala/bin/../infra/python/env/lib/python2.7/os.py", line 157, 
> in makedirs
> 02:45:52   File 
> "/home/ubuntu/Impala/bin/../infra/python/env/lib/python2.7/os.py", line 157, 
> in makedirs
> 02:45:52 mkdir(name, mode)
> 02:45:52 mkdir(name, mode)
> 02:45:52 mkdir(name, mode)
> 02:45:52 OSErrorOSErrorOSError: [Errno 17] File exists: 
> '/home/ubuntu/Impala/docker/build_context'
> 02:45:52 : [Errno 17] File exists: 
> '/home/ubuntu/Impala/docker/build_context': 
> 02:45:52 [Errno 17] File exists: '/home/ubuntu/Impala/docker/build_context'
> 02:45:52 Traceback (most recent call last):
> 02:45:52   File "./setup_build_context.py", line 60, in 
> 02:45:52 symlink_file_into_dir(so, LIB_DIR)
> 02:45:52   File "./setup_build_context.py", line 51, in symlink_file_into_dir
> 02:45:52 os.symlink(src_file, os.path.join(dst_dir, 
> os.path.basename(src_file)))
> 02:45:52 OSError: [Errno 17] File exists
> 02:45:52 docker/CMakeFiles/executor_image.dir/build.make:74: recipe for 
> target 'docker/build_context/.timestamp' failed
> 02:45:52 make[3]: *** [docker/build_context/.timestamp] Error 1
> 02:45:52 docker/CMakeFiles/catalogd_image.dir/build.make:74: recipe for 
> target 'docker/build_context/.timestamp' failed
> 02:45:52 make[3]: *** [docker/build_context/.timestamp] Error 1
> 02:45:52 CMakeFiles/Makefile2:13731: recipe for target 
> 'docker/CMakeFiles/executor_image.dir/all' failed
> 02:45:52 make[2]: *** [docker/CMakeFiles/executor_image.dir/all] Error 2
> 02:45:52 make[2]: *** Waiting for unfinished jobs
> 02:45:52 CMakeFiles/Makefile2:13665: recipe for target 
> 'docker/CMakeFiles/catalogd_image.dir/all' failed
> 02:45:52 make[2]: *** [docker/CMakeFiles/catalogd_image.dir/all] Error 2
> 02:45:52 docker/CMakeFiles/coordinator_image.dir/build.make:74: recipe for 
> target 'docker/build_context/.timestamp' failed
> 02:45:52 make[3]: *** [docker/build_context/.timestamp] Error 1
> 02:45:52 docker/CMakeFiles/coord_exec_image.dir/build.make:74: recipe for 
> target 'docker/build_context/.timestamp' failed
> 02:45:52 make[3]: *** 

[jira] [Commented] (IMPALA-8392) make -j $IMPALA_BUILD_THREADS docker_images fails with OSError: [Errno 17] File exists: '/home/ubuntu/Impala/docker/build_context'

[Tim Armstrong (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-8392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819717#comment-16819717
 ] 

Tim Armstrong commented on IMPALA-8392:
---

https://gitlab.kitware.com/cmake/cmake/issues/17585

> make -j $IMPALA_BUILD_THREADS docker_images fails with OSError: [Errno 17] 
> File exists: '/home/ubuntu/Impala/docker/build_context'
> --
>
> Key: IMPALA-8392
> URL: https://issues.apache.org/jira/browse/IMPALA-8392
> Project: IMPALA
>  Issue Type: Sub-task
>  Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Minor
>
> Looks like there is a bug in the CMake file. I didn't see this with serial 
> make or with ninja at all. I use ninja locally which is why I didn't notice 
> this while developing it.
> {noformat}
> 02:45:52 Scanning dependencies of target catalogd_image
> 02:45:52 Scanning dependencies of target coord_exec_image
> 02:45:52 Scanning dependencies of target statestored_image
> 02:45:52 Scanning dependencies of target coordinator_image
> 02:45:52 Scanning dependencies of target executor_image
> 02:45:52 [ 97%] Creating impala base build context.
> 02:45:52 [ 97%] Creating impala base build context.
> 02:45:52 [ 97%] Creating impala base build context.
> 02:45:52 [ 98%] Creating impala base build context.
> 02:45:52 [ 98%] Creating impala base build context.
> 02:45:52 Traceback (most recent call last):
> 02:45:52 Traceback (most recent call last):
> 02:45:52 Traceback (most recent call last):
> 02:45:52   File "./setup_build_context.py", line 38, in 
> 02:45:52   File "./setup_build_context.py", line 38, in 
> 02:45:52   File "./setup_build_context.py", line 38, in 
> 02:45:52 os.makedirs(OUTPUT_DIR)
> 02:45:52 os.makedirs(OUTPUT_DIR)
> 02:45:52 os.makedirs(OUTPUT_DIR)
> 02:45:52   File 
> "/home/ubuntu/Impala/bin/../infra/python/env/lib/python2.7/os.py", line 157, 
> in makedirs
> 02:45:52   File 
> "/home/ubuntu/Impala/bin/../infra/python/env/lib/python2.7/os.py", line 157, 
> in makedirs
> 02:45:52   File 
> "/home/ubuntu/Impala/bin/../infra/python/env/lib/python2.7/os.py", line 157, 
> in makedirs
> 02:45:52 mkdir(name, mode)
> 02:45:52 mkdir(name, mode)
> 02:45:52 mkdir(name, mode)
> 02:45:52 OSErrorOSErrorOSError: [Errno 17] File exists: 
> '/home/ubuntu/Impala/docker/build_context'
> 02:45:52 : [Errno 17] File exists: 
> '/home/ubuntu/Impala/docker/build_context': 
> 02:45:52 [Errno 17] File exists: '/home/ubuntu/Impala/docker/build_context'
> 02:45:52 Traceback (most recent call last):
> 02:45:52   File "./setup_build_context.py", line 60, in 
> 02:45:52 symlink_file_into_dir(so, LIB_DIR)
> 02:45:52   File "./setup_build_context.py", line 51, in symlink_file_into_dir
> 02:45:52 os.symlink(src_file, os.path.join(dst_dir, 
> os.path.basename(src_file)))
> 02:45:52 OSError: [Errno 17] File exists
> 02:45:52 docker/CMakeFiles/executor_image.dir/build.make:74: recipe for 
> target 'docker/build_context/.timestamp' failed
> 02:45:52 make[3]: *** [docker/build_context/.timestamp] Error 1
> 02:45:52 docker/CMakeFiles/catalogd_image.dir/build.make:74: recipe for 
> target 'docker/build_context/.timestamp' failed
> 02:45:52 make[3]: *** [docker/build_context/.timestamp] Error 1
> 02:45:52 CMakeFiles/Makefile2:13731: recipe for target 
> 'docker/CMakeFiles/executor_image.dir/all' failed
> 02:45:52 make[2]: *** [docker/CMakeFiles/executor_image.dir/all] Error 2
> 02:45:52 make[2]: *** Waiting for unfinished jobs
> 02:45:52 CMakeFiles/Makefile2:13665: recipe for target 
> 'docker/CMakeFiles/catalogd_image.dir/all' failed
> 02:45:52 make[2]: *** [docker/CMakeFiles/catalogd_image.dir/all] Error 2
> 02:45:52 docker/CMakeFiles/coordinator_image.dir/build.make:74: recipe for 
> target 'docker/build_context/.timestamp' failed
> 02:45:52 make[3]: *** [docker/build_context/.timestamp] Error 1
> 02:45:52 docker/CMakeFiles/coord_exec_image.dir/build.make:74: recipe for 
> target 'docker/build_context/.timestamp' failed
> 02:45:52 make[3]: *** [docker/build_context/.timestamp] Error 1
> 02:45:52 CMakeFiles/Makefile2:13797: recipe for target 
> 'docker/CMakeFiles/coord_exec_image.dir/all' failed
> 02:45:52 make[2]: *** [docker/CMakeFiles/coord_exec_image.dir/all] Error 2
> 02:45:52 CMakeFiles/Makefile2:13764: recipe for target 
> 'docker/CMakeFiles/coordinator_image.dir/all' failed
> 02:45:52 make[2]: *** [docker/CMakeFiles/coordinator_image.dir/all] Error 2
> 02:45:52 [ 98%] Building Impala base docker image.
> {noformat}
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/33/



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)