The following issue has been SUBMITTED.
======================================================================
http://public.kitware.com/Bug/view.php?id=15620
======================================================================
Reported By: James Johnston
Assigned To:
======================================================================
Project: CMake
Issue ID: 15620
Category: CMake
Reproducibility: always
Severity: major
Priority: normal
Status: new
======================================================================
Date Submitted: 2015-06-17 11:37 EDT
Last Modified: 2015-06-17 11:37 EDT
======================================================================
Summary: Ninja generator triggers race condition in Borland
bcc32 linker causing intermittent build failure
Description:
Unfortunately, it appears that the Borland linker utilizes temporary files with
a hard-coded name (e.g. turboc.$ln); these temporary files are written to the
linker's current directory. (Determined via Process Monitor). If two or more
concurrent linkers are run with the same current directory, some of them may
fail as shown below because they will both be trying to use the same temporary
filename.
The Ninja generator, unfortunately seems to keep the current directory to
CMAKE_BINARY_DIR. If the CMake project contains multiple libraries (e.g. VTK
5.4.2, which I tested and has dozens), then it is possible that Ninja will
invoke two concurrent linkers with the same current directory of
CMAKE_BINARY_DIR. They may then conflict with each other - both attempting to
write to a file named ${CMAKE_BINARY_DIR}/turboc.$ln, for example.
To compare, the NMake jom generator runs the linker for each target in a unique
target-specific directory, such that this problem never arises. If the Ninja
generator could also do this, it would work around this issue. (In fact, it
would work around any compiler that happens to write temporary files to its
current directory).
Steps to Reproduce:
My configuration:
* Quad-core hyper-threaded Core i7 CPU
* C++ Builder 5 Update 1
* VTK 5.4.2 (this version still builds under C++ Builder, unlike some newer
versions)
* CMake 3.3.0-rc1 (Kitware-provided binary)
* Ninja 1.5.3, built from source using VC++ 2008 Express
* VTK configured: CMAKE_BUILD_TYPE=RelWithDebInfo, VTK_REMOVE_LEGACY=ON,
BUILD_TESTING=OFF, BUILD_SHARED_LIBS=ON
1. Configure VTK with the above settings. (e.g. to C:\VTK-build)
2. Run "ninja" to start the build.
3. The build *may* fail if two projects happen to link concurrently. Of
course, it will always succeed if you pass "-j 1" to Ninja to only use one CPU
core.
4. Even if you don't immediately experience failure, you can use Sysinternals
Process Monitor from Microsoft to monitor the files being written on your
system. Note that you'll see repeated writes to files like
"C:\VTK-build\turboc.$ln".
Additional Information:
While I tested this with C++ Builder 5, the error appears to exist even in the
newest C++ Builder versions:
http://docwiki.embarcadero.com/RADStudio/XE8/en/E2216_Unable_to_create_turboc.$ln_(C%2B%2B)
Sample error when building VTK 5.4.2:
=====================
[214/2008] Linking CXX shared library bin\vtkDICOMParser.dll Borland C++ 5.5.1
for Win32 Copyright (c) 1993, 2000 Borland Turbo Incremental Link 5.66 Copyright
(c) 1997-2002 Borland
Fatal: No object file or .EXE name was given
Borland Implib Version 3.0.22 Copyright (c) 1991, 2000 Inprise Corporation Error
bin\: unable to open file [215/2008] Linking C shared library bin\vtkzlib.dll
FAILED: cmd.exe /C "cd . && C:\PROGRA~2\Borland\CBUILD~1\Bin\bcc32.exe -tWR
-tW- -tWD -ebin\vtkzlib.dll -tWM -lS:104857
6 -lSc:4098 -lH:1048576 -lHc:8192 -v import32.lib
Utilities\vtkzlib\CMakeFiles\vtkzlib.dir\adler32.obj <snip more OBJ files> &&
implib -c -w bin\vtkzlib.lib bin\vtkzlib.dll && cd ."
Borland C++ 5.5.1 for Win32 Copyright (c) 1993, 2000 Borland Turbo Incremental
Link 5.66 Copyright (c) 1997-2002 Borland
Fatal: Unable to open file 'turboc.$ln'
=====================
Here is the offending snippet from one of the generated ninja files -
build.ninja:
=================================================
build bin\vtkzlib.dll bin\vtkzlib.lib: C_SHARED_LIBRARY_LINKER__vtkzlib
Utilities\vtkzlib\CMakeFiles\vtkzlib.dir\adler32.obj <snip more OBJ files>
LANGUAGE_COMPILE_FLAGS = -tWM -Od
LINK_FLAGS = -tWM -lS:1048576 -lSc:4098 -lH:1048576 -lHc:8192 -v
LINK_LIBRARIES = import32.lib
POST_BUILD = cd .
PRE_LINK = cd .
TARGET_FILE = bin\vtkzlib.dll
TARGET_IMPLIB = bin\vtkzlib.lib
TARGET_PDB = vtkzlib.dll.dbg
=================================================
And the corresponding ninja rule:
=================================================
rule C_SHARED_LIBRARY_LINKER__vtkzlib
command = cmd.exe /C "$PRE_LINK &&
C:\PROGRA~2\Borland\CBUILD~1\Bin\bcc32.exe -tWR -tW- -tWD -e$TARGET_FILE
$LINK_FLAGS $LINK_PATH $LINK_LIBRARIES $in && implib -c -w $TARGET_IMPLIB
$TARGET_FILE && $POST_BUILD"
description = Linking C shared library $TARGET_FILE
restat = $RESTAT
=================================================
Notice POST_BUILD and PRE_LINK variables don't change directory to something
else. What I think will work around this bcc32 design flaw is if it can do
something like:
POST_BUILD = cd ..\..
PRE_LINK = cd Utilities\vtkzlib
(Or alternatively, cd to absolute paths like the NMake JOM example below).
And of course, adjust the relative paths in other parameters to the build rule
To contrast, NMake JOM generator has it right in order to work around this
bcc32 issue - notice we change directory to a target-specific directory before
linking, and then go back when done:
=================================================
Utilities\vtkzlib\CMakeFiles\vtkzlib.dir\build.make:
bin\vtkzlib.dll: Utilities\vtkzlib\CMakeFiles\vtkzlib.dir\adler32.obj
<snip more OBJ files>
bin\vtkzlib.dll: Utilities\vtkzlib\CMakeFiles\vtkzlib.dir\build.make
<snip CMake progress command>
cd
C:\Users\JamesJ\Documents\Repositories\SuperRepo\VTK-JOM\Utilities\vtkzlib
C:\PROGRA~2\Borland\CBUILD~1\Bin\bcc32.exe -tWR -tW- -tWD @<<
-e..\..\bin\vtkzlib.dll -tWM -lS:1048576 -lSc:4098 -lH:1048576 -lHc:8192 -v
import32.lib $(vtkzlib_OBJECTS) $(vtkzlib_EXTERNAL_OBJECTS) <<
implib -c -w ..\..\bin\vtkzlib.lib ..\..\bin\vtkzlib.dll
cd C:\Users\JamesJ\Documents\Repositories\SuperRepo\VTK-JOM
=================================================
For what it's worth, at this time, I think this issue is limited to the linker
and not to the single-unit compilation of a CPP file to OBJ file. I ran Process
Monitor for a few seconds while VTK was compiling, and it seems that the only
file writes to the build tree were either legitimate writes to named
intermediate/output files, or else to these problematic files from the linker.
======================================================================
Issue History
Date Modified Username Field Change
======================================================================
2015-06-17 11:37 James Johnston New Issue
======================================================================
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers
