Morgan Smith <[email protected]> writes: > In our build system I often specify "compile-dirty" so that I don't > trigger a world rebuild as happens with "compile". However, this is not > strictly correct. If I where to modify a macro in a file and then > "compile-dirty", then some of the "elc" files would have the older > definition of the macro.
FYI, Emacs build system struggles from the same problem. To the point that we have a very special org--inhibit-version-check just because Emacs developers say that it is "wontfix" and that broken compilation for Emacs repo is considered ok and expected with stale .elc files. > The correct way to handle this is to build out a dependency tree so that Make > is aware of exactly what ".el" files should trigger which ".elc" regeneration. > > As far as I know, for compiling, the only thing we care about is top-level > requires. So if we can pull out the top level requires from every file, we > can > build our dependency tree. I've done this using a simple regex and it seems > to > work great! With a two exceptions I've found. That's going to change. We are going to reduce top-level requires, not increase. The time to load Org is already too large, especially with debug Emacs builds. Although, for the purposes for compilation, it should be possible to leave top-level requires behind (eval-when-compile ...). > 2. For reasons beyond my comprehension, `org-babel-do-load-languages' and > `org-load-export-backends' manage to get run at compile time but I'm not > really sure what to do about that. Not sure if `org-modules' gets used at > compile time. I did not investigate. They are triggered by :set in defcustom. > However, just building a dependency tree doesn't solve all of our issues > because of the loaddefs file. The loaddefs file depends on every single > source > file and all elc files depend on the loaddefs file so we are now back where we > started with every change causing a world rebuild. And on top of that many files depend on each other to compile properly. This is also the reason why you can't build Org in isolated processes. (good luck trying to do something like make -j10 on Org) > The obvious solution is to simply not depend on loading the loaddefs file > during compilation and only load it at runtime. This means the loaddefs file > is still regenerated with every source change but that it doesn't cause a > world > rebuild. This seems to work just fine after adding in a few more `require's > and a few `declare-function's. I do not buy "seems to work". We have struggled from circular dependencies for years now. Nothing is simple in this area. And do note that it is very deliberate that we emit a very loud warning when loaddefs is not available. The fact that things work for you does not mean that they will work universally. If anything, consider the case when loaddefs is missing and Emacs will load org-loaddefs from built-in Org. That will be a disaster. > While I did put a lot of work into this change, I still feel like there is > much > I didn't investigate. I very much expect there to be some glaring flaws in > what I've done here. I do not mind improvement in this area, especially if it does not cost too much maintenance, but I am skeptical that you can solve all the edge cases. I'd say at least we need to make sure that out libraries do not have circular dependencies. That alone is a very difficult ask. > +ifeq ($(SKIP_DEPS),) > +include $(LISPF:%.el=./deps/%.d) > +else > +endif Note that these things are resolved when loading the makefile. So, trying to set SKIP_DEPS as a part of the recipe is not goiing to work. > * lisp/Makefile (autoloads): Don't clean up these files when > generating them. > * mk/targets.mk (all, compile): Don't run clean in the lisp directory > before a compilation. I would be ok to alter compile-dirty, but what you are doing here is making make compile unreliable. Your patch is adding a useful heuristics, which will fail on edge cases. -- Ihor Radchenko // yantar92, Org mode maintainer, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
