[
https://issues.apache.org/jira/browse/MESOS-10122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17095745#comment-17095745
]
Andrei Sekretenko commented on MESOS-10122:
-------------------------------------------
https://reviews.apache.org/r/72426/
https://reviews.apache.org/r/72427/
https://reviews.apache.org/r/72428/
> cmake+MSBuild is uncapable of building all Mesos sources in parallel.
> ---------------------------------------------------------------------
>
> Key: MESOS-10122
> URL: https://issues.apache.org/jira/browse/MESOS-10122
> Project: Mesos
> Issue Type: Bug
> Components: build
> Reporter: Andrei Sekretenko
> Assignee: Andrei Sekretenko
> Priority: Major
>
> When a library (in cmake's sense) contains several sources with different
> paths but the same filename (for example, slave/validation.cpp and
> resource_provider/validation.cpp), the build generated by CMake for MSVC does
> not allow for building those files in parallel (presumably, because the .obj
> files will be located in the same directory).
> This has been tested observed with both cmake 3.9 and 3.17, with "Visual
> Studio 15 2017 Win64" generator.
> It seems to be a known behaviour - see
> https://stackoverflow.com/questions/7033855/msvc10-mp-builds-not-multicore-across-folders-in-a-project.
> Two options for fixing this in a way that will work with these cmake/MSVC
> configurations are:
> - splitting the build into small static libraries (a library per directory)
> - introducing an intermediate code-generation-like step optionally
> flattening the directory structure (slave/validation.cpp ->
> slave_validation.cpp)
> Both options have their drawbacks:
> - The first will result in changing the layout the static build artifacts
> (mesos.lib will be replaced with a ton of smaller libraries), that will pose
> integration cahllenges, and potentially will result in worse parallelism.
> - The second will result in being unable to use #include without a path
> (right now there are three or four such #include's in the whole
> Mesos/libprocess code buildable on Windows) in changed value of __FILE__
> macro (as a consequence, in the example above, `validation.cpp` in logs will
> be replaced either with `slave_validation.cpp` or with
> `resource_provider_validation.cpp`)
> Note that the second approach will need to deal with potential collisions
> when the source tree has filenames with underscores. If, for example, we had
> both slave/validation.cpp and slave_validation.cpp, then either some
> additional escaping will be needed or, alternatively, such layout could be
> just forbidden (and made to fail the build).
> Preliminary testing shows that on a 8-core AWS instance flattening source
> trees of libprocess, mesos-protobufs and libmesos results in clean build
> speedup from ~1 hour to ~30 minutes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
