Michael, I'm doing this wrong too and I have expensive laptop as well. I don't know any better solution than interleave compilation with other work. This is not always productive, hence
+1 for this change. As a side note, we should probably revive the effort of a) splitting huge .cpps into smaller ones and b) moving non-template method implementations into .cpps. On Sun, Sep 25, 2016 at 3:56 PM, Michael Park <[email protected]> wrote: > Hello, > > I would like to propose a change in our build to help us improve developer > efficiency. > The proposal is to support separate compilation of unit tests. > > Currently, we have the old approach of invoking `make check -j N > GTEST_FILTER=""`, or a newer option of doing `make tests -j N`. From what > I've heard the latter is slightly faster. However, when someone is > developing a specific feature, it's likely that they would like to make > changes and iterate on a single test file. In this workflow, having to > compile (virtually) __all__ of the tests is very burdensome. This situation > is not so bad if you're working in a very isolated part of the codebase, > but it gets to be pretty bad if you're experimenting with parts that are > widely used. > > An example of a workflow I'm aiming for would look something like: > > 1. write some code... > 2. `make check master_tests` // compile and test > `src/tests/master_tests.cpp` > 3. fix compilation errors... > 4. `make check master_tests` // compile and test > `src/tests/master_tests.cpp` > 5. change some stuff... > 6. `make check master_tests` // compile and test > `src/tests/master_tests.cpp` > 7. debug... > 8. `make check master_tests` // compile and test > `src/tests/master_tests.cpp` > 9. alright, looks good. `make check` > > I have 0 attachment to the `make check master_tests` syntax. It'll be a > different syntax for CMake anyways. I just think that the ability to > perform separate compilation of tests will be immensely useful. > > Some numbers to justify what I'm proposing: > > - `make -j 8` on my laptop takes roughly 10 mins. > - `make tests -j 8` takes about 30 mins. > > Of course, not every change you make triggers all of the tests to > recompile. But if you change components that are widely used, it does end > up recompiling virtually everything. Under these circumstances, I would > love for each iteration to be 11 mins (10 mins + __at most__ 1 min to > compile the single test), rather than 30 mins. > > NOTE: My laptop is expensive... some people even use machines with 64 cores > or whatever to compile Mesos. Not everyone has access to this kind of > equipment. We should strive for something better than "throw more money at > it". > > The goal of this thread for me is the following: > (1) Capture whether this is something most people experience, or whether > I'm just doing it wrong > (2) If most people do experience this inefficiency and would like this > change to be made, > I would like to recruit 1 or 2 people to help me deliver this, since > I'm not a automake nor CMake expert. >
