Nikolay Molchanov wrote:
Ian wrote:
I typically see 70% or so improvement over parallel
with my stuff.
Wow! Are you saying that you were able to
1. create setup to run dmake in distributed mode?
2. build OpenSolaris in distributed mode?
3. and this was faster than in parallel mode?
If yes, could you *please* add more details?
Not OpenSolaris unfortunately, but my own code.
I am resp.eng. for dmake, so, of course I know,
how to use dmake in distributed mode. But I have
heard many times that our man page and dmake's
error messages are not very helpful, so users cannot
really use dmake in distributed mode. And we have
to create a tool, that will create setup to run dmake
in distributed and grid mode. What do you think
about the necessity of such tool?
I've been using dmake since it was introduced and it works best with a
flat makefile that lists one source file per target. I have found the
best tool for generating these is the good old Workshop IDE (not the
Java one!). With these makefiles, dmake in distributed mode works realy
well. With a set of individual makefiles, or makefiles that list more
than one source file per target it performs poorly or even hangs (try
building Apache with dmake).
I haven't had any problems with error messages, only the volume of them
when 10 compiles all fail! To be honest, I didn't think there was any
difference between make and dmake error messages.
If you want a god tool, revive the Workshop IDE, it works very well with
dmake. The other alternative is to produce a recursive makefile parser
that can generate on, flat, makefile for a project. The hardest part
would be sorting the order of dependencies. The Workshop IDE does a
good job of this is you gave a mix of libraries and executables in one
project. As all of my libraries have a test executable as part of their
build, this is a very usefully feature.
I currently use a mix of dual Athlon MP and Athlon64 boxes and a Pentium
M laptop, 6 CPUs in 4 machines. I tune the number of jobs per CPU
depending on the type of files, more jobs (3) with small files, less (1
or 2) with bigger ones.
Today I looked at OpenSolris source organization
and /usr/src/Makefile to find out if it makes sense
to try to build it in grid mode. I think there are at
least 3 "big" independent parts, and each of them
also have "big" independent parts, that can be
distributed to the grid systems. The first 3 "big"
parts are:
- Kernel (uts ...)
- Libraries and Commands (lib cmd)
- UCB Libraries and Commands (ucblib ucbcmd)
I'll try to modify a little Makefile to build OpenSolaris
in grid and distributed mode, and let you know.
The more targets in one makefile the better!
Ian
_______________________________________________
opensolaris-code mailing list
[email protected]
https://opensolaris.org:444/mailman/listinfo/opensolaris-code
