Still more private email copied back to the public list. This is the last one, I promise :)
I <[EMAIL PROTECTED]> wrote: > > Ah, but then now you have all the complexity of include scanning, plus all > the complexity of LD_PRELOAD. > At this point I think this solution stops looking so elegant. Since > include scanning alone is sufficient to give > good performance, why add that extra complexity? Increased generality, I > suppose, but since the performance > is going to be poor for cases which include scanning doesn't catch, it's > not clear that this is a significant win over > just falling back to local compilation for those cases, and so it's not > clear that the increased complexity is > worthwhile. IMHO. > Sascha replied: > The complexity of this is the main reason why I did not implement it :) > > The situation I have here (which is probably a bit uncommon) is a lot of > clients and a few powerful servers. All clients have a local checkout of > the code base, so the idea of caching them on the server(s) comes > naturally. Another plus of a preload+prescan solution is that it offers > nice support for PCHs. > > Based on your pre-scan solution (if it really catches all required > files), a much simpler LD_PRELOAD-based solution might be possible: > > 1. Client scans all files and compiles a list, computes all file > hashes, sends the hashes+file names to the server. > 2. Server checks the cache and queries all missing files. > 3. Client sends all queried files. > 4. Server runs the compiler in a simple preload-wrapper, that simply > serves files from the cache. If the server scans include > directories, it will only see the files transferred by the client. > If a file is not found in the cache, the wrapper will try to open > the file on the server. > > Thanks for commenting my suggestion. > I then replied: > The cacheing that you describe above is a great idea. > > But I'd rather rely on the include scanner being conservative (always > over-approximating the set of needed files) > than using an LD_PRELOAD to fetch missing files. > Sascha then replied: > Agreed. > > At that point the preload is technically no longer required. I still see > it as an option for translating the cached files to the file names as > seen on the client (eliminating the need to patch the debug info in the > generated ELF files) - but that's just an alternative to creating a > /tmp/distcc/whatever directory containing links to the cached files. > Such a preload wrapper would be really simple. > > One question remains - how do you handle nasty things like '#include > MACRO(args)'? > > A possible way to handle problematic cases like this is to simply detect > them in the pre-scan and fall back to a local preprocessor run with '-E > -H -o /dev/null' in that case. > > My PCH patch already contains an file cache implementation for PCHs > which could be generalized. I'll try to send the patch later today. > I then replied to the question "how do you handle nasty things like '#include MACRO(args)'?": The include scanner does a static analysis which gives a conservative > over-approximation of the possible values that each macro can expand to. > > This is still not perfect, because the static analysis can take exponential > time (or worse) in the worst case. > So in cases where we detect that the static analysis is taking too long, > the include server gives up, and the distcc client will then fall back to > doing local preprocessing. -- Fergus Henderson <[EMAIL PROTECTED]>
__ distcc mailing list http://distcc.samba.org/ To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/distcc