On 07/27/2010 06:48 PM, Russell Harmon wrote:
We're looking to use cmake on a project which has some dependencies
which are not properly picked up by cmake's dependency scanner. I'm not
looking to fix the scanner, because I feel the scanner is fundamentally
broken:
Yes, it certainly has some limitations. I agree that the solution is
not to fix the current scanner. I've considered using the wave c++
preprocessor library but it is problematic to locate all the implicit
include directories and preprocessor definitions for each compiler.
Using the compiler itself to get dependencies is much better.
FYI, our Fortran dependency scanner does do some limited preprocessing
to pick out correct Fortran 90 use and module directives. This is
necessary just to generate a correct build.
In order to properly scan a file for dependencies, you need to either
use the compiler specific dependency scanner [1][2], or implement a
fully featured c preprocessor. You can't do a partial job like the
current dependency scanner does.
An ancient (1990's) predecessor to CMake actually did full preprocessing
of every translation unit and scanned for the #line directives or
similar constructs in the output. The trade-off for such perfect
dependencies is that it takes almost twice as long to build because
every translation unit needs to be preprocessed twice.
We purposely chose to use a limited scanner in CMake which is very
fast and works well for many projects. Since the set of #include
lines one can grep from a source file does not depend on the
preprocessing state we can share/cache results from scanning each
file. This allows us to scan all dependencies in a target a few
orders of magnitude faster than the compiler takes to build it.
For non-developer (end-user or packager) builds that do not need
to rebuild this is very nice.
The opposing trade-off is that projects that use conditional and
macro-based includes do not get perfect dependencies. This has not
been a problem for our needs in practice. I'm not opposed to
adding support for more advanced dependency scanning in projects
that are willing to pay the cost of additional scanning time for
better dependencies. See below.
I'd encourage you to read Recursive Make Considered Harmful [3]
Ironically it is impossible to do make-time implicit dependency
scanning without recursive make (or include directive reload magic
extensions in GNU make). After generating dependencies make needs
to invoke another make recursively to load them. When we throw
generated source files into the mix then we cannot even scan them
until the sources have been generated.
1. #define ARCH x86
#define ASM_INCLUDE ARCH/asm.h
#include ASM_INCLUDE
cmake adds a dependency on ASM_INCLUDE instead of x86/asm.h
A simple test example shows me that CMake skips this dependency
rather than generating a broken build.
2. #ifdef DEBUG
#include debug_config.h
#else
#include config.h
#endif
cmake adds a dependency on both debug_config.h and config.h
In a worst-case scenario here (one which our project has), this could
generate circular dependencies which cause the build to fail.
The dependency we generate is between an object file and the
headers. The dependency lines never have .h files on the left
or .o files on the right, so there cannot be circular dependencies.
Are you actually seeing this problem with CMake?
3. #if 0
#include nonexistantfile.h
#endif
cmake adds a dependency on nonexistantfile.h when it shouldn't have
A simple test example shows me that CMake skips this dependency
because it cannot find the header's location. This is true of
system headers in some cases too.
My suggestion for a solution to this problem is to use the compiler
specific dependency generation methods.
This is a good solution, but it should be enabled optionally
for the performance reasons I describe above.
This could be done cleanly by converting dependency generation
into a plugin based system with plugins for all the supported
compilers.
Yes. Howerver they should not be plugins in the dynamic loader sense.
We do not want to support an SDK, require users to install extra
components, or depend on the target toolchain to be capable of
building a plugin that is ABI-compatible with the running CMake.
This can probably be done with a simple command-line interface.
We just need a way to tell CMake how to invoke a third-party
tool and load dependencies from the results. I've considered
doing something similar for implicit dependencies of rules
defined by add_custom_command, but that is much harder because
the VS IDE does not provide an easy way to implement it. This
is not a problem for C preprocessor dependencies though.
I envision this capability being activated by a target property
which is initialized from a CMake variable, much like the
RUNTIME_OUTPUT_DIRECTORY property. This makes it easy to
enable project-wide or in specific targets.
I
