On 06 September 2012, Chris BeHanna said:
> :nod:
>
> That performance hit is why $EMPLOYER is abandoning SCons. :-(
OT: mind if I ask what they/you are switching to? I've spent the last
day learning tup, and it has given me some interesting ideas. Curious
what else is out there that people are adopting.
> > However, something valuable is lost when you simplify dependencies so
> > dramatically. Specifically, I implemented incremental testing, which
> > was a win, but not nearly as much of a win as it could have been. E.g.
> > if you modify a source file in a low-level component that everything
> > else depends on, then you end up recompiling the world (no big deal:
> > javac is fast enough), and re-running all the tests (not nice: the
> > whole test suite is 20-30 min).
>
> Can the tests be broken down to a per-jar basis? If so, and if you
> can identify the jar or jars that changed, then you only run the tests
> for the jars that changed.
Oops, I overstated things for dramatic purposes. In fact, in the end I
had something that works pretty much as you described. Here's a
simplified version of how things looked:
common-main.jar:
com/example/utils/StringUtils.class
com/example/utils/DateUtils.class
common-test.jar:
com/example/utils/TestStringUtils.class
com/example/utils/TestDateUtils.class
guilibs-main.jar:
com/example/swingutils/CustomTable.class
guilibs-test.jar:
com/example/swingutils/TestCustomTable.class
bigapp-main.jar:
com/example/bigapp/ApplicationLogic.class
bigapp-test.jar:
com/example/bigapp/TestApplicationLogic.class
(Expand that to 75-80 jar files and ~20,000 .class files to get a feel
for how things really looked.)
The DAG looks like this:
bigapp-test.jar : source/test/com/example/bigapp/TestApplicationLogic.java
guilibs-main.jar
common-main.jar
junit.jar
javac
jar
bigapp-main.jar : source/test/com/example/bigapp/ApplicationLogic.java
guilibs-main.jar
common-main.jar
javac
jar
No .class files, just as you suggested: that's the sausage. The build
action is [Remove(classdir), Mkdir(classdir), javac -d classdir, jar].
So if I modify ApplicationLogic.java, here's what SCons does:
ApplicationLogic.java changed:
javac + jar to rebuild bigapp-main.jar
bigapp-main.jar changed:
javac + jar to rebuild bigapp-test.jar
bigapp-test.jar changed:
run com.example.bigapp.TestApplicationLogic
touch runtests/com.example.bigapp.TestApplicationLogic.stamp
Fine so far, right? Now what happens if I modify DateUtils.java?
DateUtils.java changed:
javac + jar to rebuild common-main.jar
common-main.jar changed:
javac + jar to rebuild common-test.jar
javac + jar to rebuild guilibs-main.jar
javac + jar to rebuild bigapp-main.jar
run com.example.utils.TestDateUtils
touch runtests/com.example.utils.TestDateUtil.stamp
run com.example.utils.TestStringUtils
touch runtests/com.example.utils.TestStringUtils.stamp
So you can see that TestStringUtils is run unnnecessarily. When you
modify a component with 3000 classes and 1000 tests, that's a lot of
unnnecessary test runs.
(No, we should not have had components with 3000 classes in them. I'd
say it was legacy code, but some of that code was brand new. *sigh*)
Greg
--
Greg Ward http://www.gerg.ca/
Any priest or shaman must be presumed guilty until proven innocent.
_______________________________________________
Scons-dev mailing list
Scons-dev@scons.org
http://two.pairlist.net/mailman/listinfo/scons-dev
