Re: [O-MPI devel] New build methodology
Hi Ralf Appreciate the offer, but I think at this stage it isn't worth the hassle. We either implement a long-term fix, or just pay the price. Thanks though Ralph At 01:37 AM 11/21/2005, you wrote: Hi Ralph, * Ralph H. Castain wrote on Mon, Nov 21, 2005 at 04:04:34AM CET: > Just as an FYI -- this proposed fix didn't work. It apparently takes > a great deal more to "undo" the Makefile consolidation. I worked on > it for a couple of hours and finally just gave up. I'm sorry. If you describe what did not work, I may be able to help you. This description is too thin, though. Show what you do, cut and paste the output, and maybe we can find out. I tried the proposed fix before sending it to see whether it could work. Cheers, Ralf ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [O-MPI devel] New build methodology
Yeah, I recall the discussions and we did, in fact, bring the subdirectory thing up. What I'm now seeing is that, at least for me, I hadn't really realized the full implications of that decision. It works great if you are just making changes within an existing subsystem - but if you need to modify an API, you are in a world of hurt. I don't know if this tradeoff is good or not - I just thought it worth noting for consideration since I hadn't understood the implications as thoroughly when we had the original discussions as I now do. It seems to me that the "ideal" solution would be to provide a switch that let's me not couple all these systems into one big "make". Not sure how hard that would be to do since I have no great understanding of the build system anyway. Failing that, it might be nice to have a "developers" build somewhere that makes changes easier to make - then we can transition those to this more streamlined build for others to use. Again, not sure of the right answer. Just pointing out that this faster build system came with a price. Ralph At 08:48 PM 11/20/2005, you wrote: Well, we discussed this issue at fairly great length at the various meetings, and every time the answer was "make the build system faster". I believe we even had a discussion about this very thing (not being able to build in subdirectories). This weekend's change should have almost no real affect on the ability to add/modify/debug anything. Before, if you ran "make" in a framework, it recursed into the base/ directory, ran make there, then ran make in the framework directory (which never did anything). It didn't build any of the components in that framework (it hasn't for a month or so). Now, if you run "make" in a framework directory, it just builds the stuff in base without recusing. Of course, you can't run make in the base/ directory, but since running make in the framework directory is essentially equivalent, it doesn't exactly matter. Brian On Nov 20, 2005, at 10:04 PM, Ralph H. Castain wrote: > Just as an FYI -- this proposed fix didn't work. It apparently takes > a great deal more to "undo" the Makefile consolidation. I worked on > it for a couple of hours and finally just gave up. > > The last commits, of course, only made this situation worse as even > more of the tree now is so heavily integrated that you really can't > work on just one subsystem any more - the whole tree has to be > corrected if you change an API before you can test your changes. > > Frankly, it isn't clear to me that we really improved things by all > these changes. We may have made an autogen faster, but we have > definitely made it harder to develop a subsystem. Is that really a > good trade? I wonder. > > Ralph > > > > At 08:08 AM 11/15/2005, you wrote: >> * Ralph H. Castain wrote on Tue, Nov 15, 2005 at 03:45:26PM CET: >>> At 07:33 AM 11/15/2005, you wrote: Would it help if only the change not to build a convenience archive in orte/dps would be reverted? You could then cd orte make dps/libdps.la and would only have to issue the link command for liborte.la manually (to override rebuilding all other files that depend on dps.h). This change has very little impact on overall autogen/build execution time. >>> >>> Your proposed change would help a great deal - thanks! Can you steer >>> me through the change? >> >> Sure. Make sure to have automake installed, apply the patch below, >> then `make' should cause the Makefiles to be rebuilt. >> >> For building/testing, I'd do as above: >> cd orte >> make dps/libdps.la >> >> To now recreate liborte.la without triggering all other stuff: >> If you happen to use GNU make, you could use this hack (warning! >> hack! do not ever do this for non-debug builds!) >> make -t >> This just touches everything, so the next run of make will believe >> everything to be up to date. (You need a "make clean" to undo this >> when you really want everything updated.) >> >> Then, to recreate liborte, you do >> ../libtool --mode=clean rm -f liborte.la >> make liborte.la >> >> (In most but not all cases, the former can be replaced by "rm -f >> liborte.la"). Hope that helps. >> >> Cheers, >> Ralf >> >> Index: orte/dps/Makefile.am >> === >> --- orte/dps/Makefile.am(revision 8159) >> +++ orte/dps/Makefile.am(working copy) >> @@ -24,7 +24,8 @@ >> dps/dps_types.h \ >> dps/dps_internal.h >> >> -liborte_la_SOURCES += \ >> +noinst_LTLIBRARIES += dps/libdps.la >> +dps_libdps_la_SOURCES = \ >> dps/dps_internal_functions.c \ >> dps/dps_load_unload.c \ >> dps/dps_lookup.c \ >> Index: orte/Makefile.am >> === >> --- orte/Makefile.am(revision 8159) >> +++ orte/Makefile.am(working copy) >> @@ -33,6 +33,7 @@ >> lib_LTLIBRARIES =
Re: [O-MPI devel] New build methodology
Just as an FYI -- this proposed fix didn't work. It apparently takes a great deal more to "undo" the Makefile consolidation. I worked on it for a couple of hours and finally just gave up. The last commits, of course, only made this situation worse as even more of the tree now is so heavily integrated that you really can't work on just one subsystem any more - the whole tree has to be corrected if you change an API before you can test your changes. Frankly, it isn't clear to me that we really improved things by all these changes. We may have made an autogen faster, but we have definitely made it harder to develop a subsystem. Is that really a good trade? I wonder. Ralph At 08:08 AM 11/15/2005, you wrote: * Ralph H. Castain wrote on Tue, Nov 15, 2005 at 03:45:26PM CET: > At 07:33 AM 11/15/2005, you wrote: > > > >Would it help if only the change not to build a convenience archive in > >orte/dps would be reverted? You could then > > cd orte > > make dps/libdps.la > > > >and would only have to issue the link command for liborte.la manually > >(to override rebuilding all other files that depend on dps.h). This > >change has very little impact on overall autogen/build execution time. > > Your proposed change would help a great deal - thanks! Can you steer > me through the change? Sure. Make sure to have automake installed, apply the patch below, then `make' should cause the Makefiles to be rebuilt. For building/testing, I'd do as above: cd orte make dps/libdps.la To now recreate liborte.la without triggering all other stuff: If you happen to use GNU make, you could use this hack (warning! hack! do not ever do this for non-debug builds!) make -t This just touches everything, so the next run of make will believe everything to be up to date. (You need a "make clean" to undo this when you really want everything updated.) Then, to recreate liborte, you do ../libtool --mode=clean rm -f liborte.la make liborte.la (In most but not all cases, the former can be replaced by "rm -f liborte.la"). Hope that helps. Cheers, Ralf Index: orte/dps/Makefile.am === --- orte/dps/Makefile.am(revision 8159) +++ orte/dps/Makefile.am(working copy) @@ -24,7 +24,8 @@ dps/dps_types.h \ dps/dps_internal.h -liborte_la_SOURCES += \ +noinst_LTLIBRARIES += dps/libdps.la +dps_libdps_la_SOURCES = \ dps/dps_internal_functions.c \ dps/dps_load_unload.c \ dps/dps_lookup.c \ Index: orte/Makefile.am === --- orte/Makefile.am(revision 8159) +++ orte/Makefile.am(working copy) @@ -33,6 +33,7 @@ lib_LTLIBRARIES = liborte.la liborte_la_SOURCES = liborte_la_LIBADD = \ + dps/libdps.la \ $(MCA_orte_FRAMEWORK_LIBS) \ $(top_ompi_builddir)/opal/libopal.la liborte_la_DEPENDENCIES = $(liborte_la_LIBADD) ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [O-MPI devel] New build methodology
Your proposed change would help a great deal - thanks! Can you steer me through the change? At 07:33 AM 11/15/2005, you wrote: Hi Ralph, * Ralph H. Castain wrote on Tue, Nov 15, 2005 at 03:12:38PM CET: > > While I generally find the new build methodology (i.e., reducing the > number of makefiles) has little impact on me, I have now encountered > one problem that causes a significant difficulty. In trying to work > on a revised data packing system for the orte part of the branch, I > now find that I cannot compile and debug the dps *until* I have > completely revised all the rest of the system that uses it. In other > words, having made a change to the dps, I first have to go through > every function that uses it and make the tree conform BEFORE I can > even begin to debug and test the revisions in the dps itself. Would it help if only the change not to build a convenience archive in orte/dps would be reverted? You could then cd orte make dps/libdps.la and would only have to issue the link command for liborte.la manually (to override rebuilding all other files that depend on dps.h). This change has very little impact on overall autogen/build execution time. > The problem this creates is that - unless I am absolutely correct in > my first iteration on the changes - I find myself repeatedly going > through the tree, modifying the API calls into the dps, getting > everything to compile, then trying the dps, .discovering that I > need to make a change, going through the entire tree to modify > everything again, trying the change,. Hmm. For one you are seeing the effect of what is actually a more precise representation of the dependencies; though I agree that during development this can hurt. > Perhaps an option to create a local makefile would help? Not sure if > that is possible, but it sure would ease my pain! Hmm. May be possible with some hackery, but if above would work well enough for you, it'd be much easier. Thanks for the feedback by the way, I hadn't thought of this. Cheers, Ralf ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [O-MPI devel] New build methodology
Hi Ralph, * Ralph H. Castain wrote on Tue, Nov 15, 2005 at 03:12:38PM CET: > > While I generally find the new build methodology (i.e., reducing the > number of makefiles) has little impact on me, I have now encountered > one problem that causes a significant difficulty. In trying to work > on a revised data packing system for the orte part of the branch, I > now find that I cannot compile and debug the dps *until* I have > completely revised all the rest of the system that uses it. In other > words, having made a change to the dps, I first have to go through > every function that uses it and make the tree conform BEFORE I can > even begin to debug and test the revisions in the dps itself. Would it help if only the change not to build a convenience archive in orte/dps would be reverted? You could then cd orte make dps/libdps.la and would only have to issue the link command for liborte.la manually (to override rebuilding all other files that depend on dps.h). This change has very little impact on overall autogen/build execution time. > The problem this creates is that - unless I am absolutely correct in > my first iteration on the changes - I find myself repeatedly going > through the tree, modifying the API calls into the dps, getting > everything to compile, then trying the dps, .discovering that I > need to make a change, going through the entire tree to modify > everything again, trying the change,. Hmm. For one you are seeing the effect of what is actually a more precise representation of the dependencies; though I agree that during development this can hurt. > Perhaps an option to create a local makefile would help? Not sure if > that is possible, but it sure would ease my pain! Hmm. May be possible with some hackery, but if above would work well enough for you, it'd be much easier. Thanks for the feedback by the way, I hadn't thought of this. Cheers, Ralf
