[kbuild-devel] Some feedback on using kbuild
Keith,
thanks for your previous reply.
( Re: kbuild2.5 does not actually install vmlinux for debugging as claimed )
Here are some more questions/observations on using kbuild : I hope
this kind of feedback is useful.
1) what version of make ? Makefile-2.5 suggests I need at least 3.79.1
but kbuild-2.5.txt says 3.7[78] will work ( section CONTROLLING KBUILD ).
2) $USER_VERSION versus $USERVERSION ?
Documentation/kbuild/kbuild-2.5.txt refers to USER_VERSION, but
Makefile-2.5 uses USERVERSION. Also, kbuild-2.5.txt refers to
file .tmp_A_version which does not seem to exist. Assuming
.tmp_A_version is not required, this is my take on USERVERSION
--- 2.5-kbuild-src/Documentation/kbuild/kbuild-2.5.txt Wed Jun 19 12:17:11 2002
+++ 2.5-kbuild-src/Documentation/kbuild/kbuild-2.5.txt.jak Sun Jun 23 11:39:43
+2002
@@ -1994,6 +1994,22 @@
forces a complete rebuild. 'arch_opt' is a set of architecture specific
config variables, such as a list of machine types.
+ USERVERSION
+In current kernels the kernel version information is part of the top
+level Makefile. I want to move them to a separate file so the top level
+Makefile it is only changed for kbuild, not for every kernel. The plan
+is to ship a file A_version containing just the VERSION, PATCHLEVEL,
+SUBLEVEL and EXTRAVERSION variables with the kernel sources.
+In the meantime, we generate our own $KBUILD_OBJTREE/A_version containing
+these variables based on the old Makefile.
+
+Variable USERVERSION is _not_ part of A_version, this variable is
+reserved for the end user and must never be set in any distributed kernel.
+The user can then customize the kernel version by setting USERVERSION as
+part of the make command line e.g.
+ make -f $KBUILD_SRCTREE_000/Makefile-2.5 -j2 USERVERSION=whatever
+The USERVERSION text gets appended to the existing kernel version info.
+
TEMPORARY FILES FOR KBUILD
@@ -2013,17 +2029,6 @@
modules, converting vmlinix to a bootable format, installing the bootable
kernel, installing the modules and running the post install script, if
any.
-
- .tmp_A_version
-In current kernels the kernel version information is part of the top
-level Makefile. I want to move them to a separate file so the top level
-Makefile it is only changed for kbuild, not for every kernel. The plan
-is to create file A_version containing just the VERSION, PATCHLEVEL,
-SUBLEVEL and EXTRAVERSION variables. In the meantime, create
-.tmp_A_version from the old Makefile and compare against the generated
-A_version file. Variable USER_VERSION is _not_ part of A_version, this
-variable is reserved for the end user and must never be set in any
-distributed kernel.
.tmp_db_main
The main kbuild database. This uses Larry McVoy's memory mapped database
### end
3) what are the legal options for using shadow tree(s) e.g.
what is the proper procedure for switching between using shadow tree(s),
then reverting to Linus sources. It looks like I need to do make mrproper and full
recompile,
even though I am only ever likely to be changing/adding/dropping compilation of a small
number of objects. I would have expected the "only rebuild what's necessary"
philosophy to apply here.
I think shadow tree support is a feature that will impress lots of people most, and
deserves
more detailed description/instructions.
4) make *config insists I must do make dep
( I am trying kbuild-*-2.4.19-pre10
while 2.5.x plays hard to compile ).
--- 2.4-src/scripts/Configure Mon Jun 17 16:36:27 2002
+++ 2.4-src/scripts/Configure.jak Tue Jun 18 22:42:18 2002
@@ -696,7 +696,7 @@
echo "*** End of Linux kernel configuration."
echo "*** Check the top-level Makefile for additional configuration."
if [ ! -f .hdepend -o "$CONFIG_MODVERSIONS" = "y" ] ; then
-echo "*** Next, you must run 'make dep'."
+echo "*** Next, you must run 'make dep' ( unless you are using kbuild-2.5 )."
else
echo "*** Next, you may run 'make bzImage', 'make bzdisk', or 'make install'."
fi
--- 2.4-src/scripts/Menuconfig Mon Jun 10 14:17:11 2002
+++ 2.4-src/scripts/Menuconfig.jak Tue Jun 18 22:43:23 2002
@@ -1454,7 +1454,7 @@
echo "*** End of Linux kernel configuration."
echo "*** Check the top-level Makefile for additional configuration."
if [ ! -f .hdepend -o "$CONFIG_MODVERSIONS" = "y" ] ; then
- echo "*** Next, you must run 'make dep'."
+ echo "*** Next, you must run 'make dep' ( unless you are using kbuild-2.5
+)."
else
echo "*** Next, you may run 'make bzImage', 'make bzdisk', or 'make
install'."
fi
--- 2.4-src/scripts/header.tk Mon Jul 2 21:56:40 2001
+++ 2.4-src/scripts/header.tk.jak Tue Jun 18 22:44:31 2002
@@ -551,7 +551,7 @@
global CONFIG_MODVERSIONS; vfix CONFIG_MODVERSIONS
if { ([file exists .hdepend] != 1) || ($CONFIG_MODVERSIONS == 1) } the
Re: [kbuild-devel] Some feedback on using kbuild
On Sun, 23 Jun 2002 12:14:59 +0100, Jak <[EMAIL PROTECTED]> wrote: >Here are some more questions/observations on using kbuild : I hope >this kind of feedback is useful. > >1) what version of make ? Makefile-2.5 suggests I need at least 3.79.1 >but kbuild-2.5.txt says 3.7[78] will work ( section CONTROLLING KBUILD ). I tried to keep to 3.77 for compatibility with the 2.4 kernel (only requires 3.77) but had to abandon that aim. The recommended version of make is 3.79, I will update the docs. >2) $USER_VERSION versus $USERVERSION ? >Documentation/kbuild/kbuild-2.5.txt refers to USER_VERSION, but >Makefile-2.5 uses USERVERSION. Also, kbuild-2.5.txt refers to >file .tmp_A_version which does not seem to exist. Assuming >.tmp_A_version is not required, this is my take on USERVERSION It should be USERVERSION, not USER_VERSION. .tmp_A_version used to exist but no longer. Documentation updated. >3) what are the legal options for using shadow tree(s) e.g. > what is the proper procedure for switching between using shadow tree(s), >then reverting to Linus sources. It looks like I need to do make mrproper and full >recompile, >even though I am only ever likely to be changing/adding/dropping compilation of a >small >number of objects. I would have expected the "only rebuild what's necessary" >philosophy to apply here. That is what should occur. Adding or deleting a shadow tree should only rebuild based on the files that were added, deleted, shadowed or unshadowed. If something else is occurring, I need the details. >I think shadow tree support is a feature that will impress lots of people most, and >deserves >more detailed description/instructions. Linus and Kai do not think that shadow trees are useful. I have given up explaining why they are useful. Now I am waiting until Kai hits a dead end using the current syntax (there is no way that the current syntax can cope with shadow trees) then I will try to raise some interest in using the new syntax to get the advanced features of kbuild 2.5. >4) make *config insists I must do make dep >( I am trying kbuild-*-2.4.19-pre10 >while 2.5.x plays hard to compile ). Compatibility glue with kbuild 2.4. They use the same scripts for make *config and dep is required for kbuild 2.4. >5) I can't figure out how to make the kernel docs now. I can see a Makefile.in in >Documentation/DocBook, >so there must be a way ! >There should probably be some scripts or instructions for each of the old Makefile >targets now evicted >from Makefile-2.5. make one of doc-books, doc-ps, doc-pdf, doc-html. >6) Finally, in the "file this under curiosities department", I have seen the >following error output a couple of times : >make -f $KBUILD_SRCTREE_000/Makefile-2.5 -j2 HOSTCC=gcc-3.1 CC=gcc-3.1 >Using ARCH='i386' AS='as' LD='ld' CC='gcc-3.1' CPP='gcc-3.1 -E' AR='ar' HOSTAS='as' >HOSTLD='gcc' HOSTCC='gcc-3.1' HOSTAR='ar' >Generating global Makefile > phase 1 (find all inputs) >/home/compo/2.4-src/scripts/pp_makefile4.c: In function `read_tmp_select': >/home/compo/2.4-src/scripts/pp_makefile4.c:686: warning: `eof' might be used >uninitialized in this function >/home/compo/2.4-src/scripts/pp_makefile4.c: In function `read_makefile': >/home/compo/2.4-src/scripts/pp_makefile4.c:781: warning: `eof' might be used >uninitialized in this function >/home/compo/2.4-src/scripts/pp_makefile4.c: In function `copy_makefile': >/home/compo/2.4-src/scripts/pp_makefile4.c:2711: warning: `eof' might be used >uninitialized in this function > phase 2 (convert all Makefile.in files) > phase 3 (evaluate selections) > phase 4 (integrity checks, write global makefile) >Starting phase 5 (build) for installable gcc 3.1 being a little too smart for its own good, the warnings are spurious. Changing int ... eof, to int ... eof=0, will avoid those warnings. >This happens sometimes if I do make mrproper; cp saved.config $KBUILD_OBJTREE/.config >; make oldconfig installable >It doesn't seem to cause any problems, and the .config file is OK thereafter. The >saved.config file may be from a previous >kernel version, so maybe it has some invalid config ? Probably a "don't do that !", >but saving a .config file and copying >it to $KBUILD_OBJTREE is referred to in kbuild-2.5.txt. The gcc warnings have nothing to do with copying configs around, what you are doing is perfectly valid. make mrproper deletes everything including pp_makefile4, the warnings occur with gcc 3.1 on every build of pp_makefile4. >Hey, new question, in the command line in above example, I specified CC and HOSTCC. >CPP becomes $(CC) -E, >whereas HOSTLD stays as gcc. So I need to specify 3 command line variables CC, HOSTCC >& HOSTLD to avoid >using whatever compiler gcc happens to point at. Is HOSTLD = gcc when HOSTCC != gcc >correct ? Correct but unexpected. Using gcc as a replacement for HOSTLD overcomes problems with extra flags that LD does not recognise. I will change HOSTLD to default to HOSTCC. It probably
Re: [kbuild-devel] Some feedback on using kbuild
On Sun, Jun 23, 2002 at 11:54:29PM +1000, Keith Owens wrote: > Linus and Kai do not think that shadow trees are useful. I have given > up explaining why they are useful. Now I am waiting until Kai hits a > dead end using the current syntax (there is no way that the current > syntax can cope with shadow trees) then I will try to raise some > interest in using the new syntax to get the advanced features of kbuild > 2.5. Hi Keith. To me the functionality provided by kbuild-2.5 with respect to shadow trees belongs to the SCM system. What kbuild-2.5 address is only the simplest part namely the compile step. A proper SCM system allow you to do parrallel development with a seperate integration branch, or whatever term the SCM in question like to use. Why do you see so much added value in kbuild support for shadow trees compared to what a proper SCM tool give you? Personally I do not see this feature as important enough to justify a new syntax for all makefiles in the tree. Sam --- Sponsored by: ThinkGeek at http://www.ThinkGeek.com/ ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
On Sunday 23 June 2002 22:14, Sam Ravnborg wrote: > To me the functionality provided by kbuild-2.5 with respect to shadow > trees belongs to the SCM system. > What kbuild-2.5 address is only the simplest part namely the compile step. > A proper SCM system allow you to do parrallel development with a > seperate integration branch, or whatever term the SCM in question > like to use. > > Why do you see so much added value in kbuild support for shadow trees > compared to what a proper SCM tool give you? For my (linux on s390) purpose, shadow trees are the most important feature of kb25, because they allow us to use a proper SCM at all. Currently, we use a CVS repository that includes the official Linux tree as well as our closed source drivers. I would love to use Bitkeeper, but that's only possible if we can easily seperate the free from the the nonfree stuff without having two complete (incompatible) repositories. Shadow trees allow just that. For most other people, the added value is that they don't have to use the same SCM tool as anyone else. Giving the user a tarball with a driver (or multiple ones, for that matter) is just so much easier than telling him/her how to patch the source with all things that can go wrong there. Arnd <>< --- Sponsored by: ThinkGeek at http://www.ThinkGeek.com/ ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
On Mon, Jun 24, 2002 at 12:41:28AM +0200, Arnd Bergmann wrote: > > Why do you see so much added value in kbuild support for shadow trees > > compared to what a proper SCM tool give you? > > For my (linux on s390) purpose, shadow trees are the most important > feature of kb25, because they allow us to use a proper SCM at all. > Currently, we use a CVS repository that includes the official Linux tree > as well as our closed source drivers. I would love to use Bitkeeper, but > that's only possible if we can easily seperate the free from the the > nonfree stuff without having two complete (incompatible) repositories. > Shadow trees allow just that. So basically you need a tool to merge two directory structures without touching unchanged files. You could say that this is what you get for free in kbuild-2.5. But I do not see this as part of the kernel build system. Do you need this feature then create a small script that do exactly what you need and run it before the build process. The script should basically just copy modified files to the build tree. My point is that the kernel build system should not be cluttered with functionality that belongs in supporting tools. Hmm, and I did not see why you could not use BitKeeper or some other SCM for both the free anf non-free stuff?? A matter of dollars to spend? > For most other people, the added value is that they don't have to use > the same SCM tool as anyone else. Giving the user a tarball with > a driver (or multiple ones, for that matter) is just so much easier > than telling him/her how to patch the source with all things that can > go wrong there. When the functionality in question touches common files this approach does no longer work. Therefore keep half solution out of the kernel build system. Sam --- Sponsored by: ThinkGeek at http://www.ThinkGeek.com/ ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
On Sun, 23 Jun 2002 22:14:58 +0200, Sam Ravnborg <[EMAIL PROTECTED]> wrote: >On Sun, Jun 23, 2002 at 11:54:29PM +1000, Keith Owens wrote: > >> Linus and Kai do not think that shadow trees are useful. I have given >> up explaining why they are useful. Now I am waiting until Kai hits a >> dead end using the current syntax (there is no way that the current >> syntax can cope with shadow trees) then I will try to raise some >> interest in using the new syntax to get the advanced features of kbuild >> 2.5. > >Hi Keith. >To me the functionality provided by kbuild-2.5 with respect to shadow >trees belongs to the SCM system. >What kbuild-2.5 address is only the simplest part namely the compile step. >A proper SCM system allow you to do parrallel development with a >seperate integration branch, or whatever term the SCM in question >like to use. Compiling from an SCM system only works if everybody uses the same SCM tool, that is not an option for the kernel. kbuild 2.5 works for everyone, no matter what other tools the developer uses. Some architecture dependent code is not in Linus's tree because it requires changes to common code that will break other architectures. For a long time you could not apply the ia64 patch to the kernel then compile for other architectures, the other architectures would break. Merging change sets for arch dependent code is a problem for any SCM. I was using PRCS for my kernel work long before BK came along, it has patch sets and branches with good tools for merging changes together. Larry has said that PRCS is one of the best SCM's other than BK, it is only the lack of a distributed repository that stopped PRCS being used more widely. Even with a decent SCM like PRCS, handling multiple patches and testing all the combinations is a nightmare. I was working on ia64, xfs and kdb and had to ensure that these patch combinations worked :- Standard kernel Standard + ia64 Standard + ia64 + xfs Standard + ia64 + kdb(ia64) Standard + ia64 + xfs + kdb Standard + xfs Standard + kdb(i386) Standard + xfs + kdb Without shadow trees, I had to maintain 8 separate source trees to support those combinations. Any xfs change had to be manually replicated over 4 trees, any ia64 change had to manually replicated over 4 other trees. Any change to the standard kernel had to be manually replicated over all 8 trees. kdb was even worse, it consists of arch independent patches plus arch dependent patches. kdb has to be done that way because some of the kdb arch patches go over other arch patches that are not in base kernel. Add devfs to that mix and you need at least another 7 trees. Manually tracking and replicating changes across multiple source trees takes far too much time and is error prone. Using an SCM introduces its own set of problems. You are hacking away, you decide that a change was a mistake and you want to revert to the SCM version of that file. In some (most?) SCM systems, the checkout process on a file resets the timestamp to when it was checked in. Standard 'make' processing cannot cope, it expects timestamps to always go forward, not backwards. kbuild 2.5 tracks all timestamp changes. Another set of problems with an SCM is separating the change sets when you are satisfied with the updates and you do the check in. With two or more change sets in the same source tree, how do you do a check in as separate change sets? Once you mingle two change sets in the same source tree, splitting again is extremely messy and error prone. It is far better to keep them separate in the first place. An SCM might work for just one set of changes (ignoring the checkout timestamp problem). It falls down when you are working on multiple change sets, expecially when they are arch dependent. When shadow trees were first suggested I argued against them, but experience has shown that they are better than an SCM system. For ia64 + xfs + kdb I keep one copy of the sources for the standard kernel, ia64, xfs and kdb (i386 and ia64) then use shadow trees to mix and match all the combinations, each combination has its own object tree. Each change set is kept separate and can be checked in and out separately, without worrying about timestamp problems or manual replication of changes. --- Sponsored by: ThinkGeek at http://www.ThinkGeek.com/ ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
Hi, On Mon, 24 Jun 2002, Keith Owens wrote: > Even with a decent SCM like PRCS, handling multiple patches and testing > all the combinations is a nightmare. I was working on ia64, xfs and > kdb and had to ensure that these patch combinations worked :- If you just want to test patches, a "cp -lra + patch" should do the same job. Managing these patches on the other hand is better left to a SCM. bye, Roman --- Sponsored by: ThinkGeek at http://www.ThinkGeek.com/ ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
On Mon, Jun 24, 2002 at 08:49:34AM +1000, Keith Owens wrote: > Compiling from an SCM system only works if everybody uses the same SCM > tool, that is not an option for the kernel. kbuild 2.5 works for > everyone, no matter what other tools the developer uses. This does not make sense. The kernel is distributed in two ways today. As diff patches as usual and as BitKeeper repositories. The diff patch version allow you with some tweaking to import new kernels to virtually all SCM systems that is suitable for kernel development. There is absolutely no requirement that all kernel developers uses the same SCM system. Obviously the kernel build system should work for everyone irrespective of the SCM system in use. This put at least the following demands: 1) Separate OBJ and SRC tree 2) That kbuild does not touch any files in the SRC tree [Some SCM systems likes it like this, others are less restrictive. Some developers likes it likes this as well]. When starting to use a SCM system the developer gets all the functionality provided by such a tool with respect to parrallel development etc. [snip] > Standard kernel > Standard + ia64 > Standard + ia64 + xfs > Standard + ia64 + kdb(ia64) > Standard + ia64 + xfs + kdb > Standard + xfs > Standard + kdb(i386) > Standard + xfs + kdb > > Without shadow trees, I had to maintain 8 separate source trees to > support those combinations. Any xfs change had to be manually > replicated over 4 trees, any ia64 change had to manually replicated > over 4 other trees. Any change to the standard kernel had to be > manually replicated over all 8 trees. > > kdb was even worse, it consists of arch independent patches plus arch > dependent patches. kdb has to be done that way because some of the kdb > arch patches go over other arch patches that are not in base kernel. > > Add devfs to that mix and you need at least another 7 trees. Manually > tracking and replicating changes across multiple source trees takes far > too much time and is error prone. All the above look like ordinary problems for me, and we can only agree that some kind of tool support is needed during daily work with this amount of different trees. So the point for discussion is solely: a) Where is this functionality best located b) How to implement this I have seen NO discussion of the above items so far. Therefore I bring it up now together with my opinions. Shadow tree functionality = The functionality to support shadow trees are best located in an SCM tool. If the developer or group in question does not use a SCM tool - or maybe a non-coherent setup with two different SCM tools then there is a need to support this at another place. The best place is in between the kbuild and the trees managed. How to implement shadow trees = within the SCM tool. It no SCM tool is used then a simple script that takes all updates from one or more separate SRC trees and copy them to a common place could do it. Then combine this with a wrapper around kbuild, and viola, support for shadow trees. No magic in kbuild, no need to complicate a tool used by so many others. > Using an SCM introduces its own set of problems. You are hacking away, > you decide that a change was a mistake and you want to revert to the > SCM version of that file. In some (most?) SCM systems, the checkout > process on a file resets the timestamp to when it was checked in. I have worked with the following SCM systems: RCS, Clearcase, BitKeeper None of them had this BUG! They had the possibility to do it, but it was clearly mentioned in the Documentation that it could confuse make. There is no reason to add a lot of complexibility to kbuild, to cover malconfiguration in SCM tools. > Standard 'make' processing cannot cope, it expects timestamps to always > go forward, not backwards. kbuild 2.5 tracks all timestamp changes. > > Another set of problems with an SCM is separating the change sets when > you are satisfied with the updates and you do the check in. With two > or more change sets in the same source tree, how do you do a check in > as separate change sets? Once you mingle two change sets in the same > source tree, splitting again is extremely messy and error prone. It is > far better to keep them separate in the first place. In Cleacase I just establish two different views. No problem. I BitKeeper I just execute bk clone twice. No problem. So I do not recognize the problems that you describe above. But it could be me that is unfamilar with CVS, PRCS, arch, ??? > An SCM might work for just one set of changes (ignoring the checkout > timestamp problem). It falls down when you are working on multiple > change sets, expecially when they are arch dependent. > > When shadow trees were first suggested I argued against them, but > experience has shown that they are better than an SCM system. For ia64 > + xfs + kdb I keep one copy of the sources for the standard kern
Re: [kbuild-devel] Some feedback on using kbuild
G'day, I'll just briefly unlurk on the subject of kbuild before getting back to the kconfig bughunt. Sam Ravnborg wrote: > > On Mon, Jun 24, 2002 at 08:49:34AM +1000, Keith Owens wrote: > > [...] > There is absolutely no requirement that all kernel developers uses > the same SCM system. Agreed. But this means that when dealing with the kernel build system we need to plan for the case where someone is working in a workarea built from a combination of two or more trees, partial trees, or patches from different sources kept in different SCMs. So we need to plan for the lowest common denominator in SCM systems, which is none at all i.e. getting a bunch of tarballs from ftp.kernel.org and elsewhere. > Obviously the kernel build system should work for everyone irrespective > of the SCM system in use. This put at least the following demands: > 1) Separate OBJ and SRC tree > 2) That kbuild does not touch any files in the SRC tree Yes, this is the absolute minimum useful behaviour for a build system. For example autoconf/automake work this way in user space. Note that kbuild 2.4 doesn't work this way (or at least it didn't before Kai got started). > > Without shadow trees, I had to maintain 8 separate source trees to > > support those combinations. [...] Manually > > tracking and replicating changes across multiple source trees takes far > > too much time and is error prone. > All the above look like ordinary problems for me, and we can > only agree that some kind of tool support is needed during daily work > with this amount of different trees. > > So the point for discussion is solely: > a) Where is this functionality best located > b) How to implement this > > I have seen NO discussion of the above items so far. Therefore I bring it > up now together with my opinions. Fair enough. > Shadow tree functionality > = > The functionality to support shadow trees are best located in an SCM tool. > If the developer or group in question does not use a SCM tool - or maybe > a non-coherent setup with two different SCM tools then there is a need > to support this at another place. > The best place is in between the kbuild and the trees managed. Sure, this is basically an SCM problem, and in an ideal world would be handled by the SCM tool and kbuild itself would just see a unified view of files. However in the real world we need to plan for the case where the SCM is too dumb to do this and too (ahem) precious to be fixed to do this, or even nonexistant, so we end up having to do it in kbuild. This is the same reason why we have the extraordinary complexity of autoconf and automake for usermode programs: it's regrettably necessary to deal with imperfect and unfixable platforms. > How to implement shadow trees > = > within the SCM tool. > It no SCM tool is used then a simple script that takes all updates > from one or more separate SRC trees and copy them to a common place > could do it. And then afterwards we've got to untangle this mess again, completely reliably and automatically. The beauty of having proper shadow trees is that the trees are kept completely separate at all times except in the compiler. There is *no* danger of ending up sending the wrong patch to the wrong place or checking the wrong file into the wrong SCM. > > [...] In some (most?) SCM systems, the checkout > > process on a file resets the timestamp to when it was checked in. > I have worked with the following SCM systems: > RCS, Clearcase, BitKeeper > None of them had this BUG! CVS does this. Maybe there's a way to turn it off, I don't know. > > Another set of problems with an SCM is separating the change sets when > > you [...] do the check in. [...] > In Cleacase I just establish two different views. No problem. > I BitKeeper I just execute bk clone twice. No problem. > So I do not recognize the problems that you describe above. > But it could be me that is unfamilar with CVS, PRCS, arch, ??? It is a problem with every SCM too dumb to know about change sets, in particular CVS. Remember, CVS is used by everyone too poor for ClearCase and too cautious or oldfashioned for BK. > > When shadow trees were first suggested I argued against them, but > > experience has shown that they are better than an SCM system. I agree with Keith, shadow trees rock. I think they are probably the single most useful feature of kbuild 2.5. I fervently hope we end up with shadow trees or something like them by the end of Linux 2.5. Greg. -- the price of civilisation today is a courageous willingness to prevail, with force, if necessary, against whatever vicious and uncomprehending enemies try to strike it down. - Roger Sandall, The Age, 28Sep2001. --- Sponsored by: ThinkGeek at http://www.ThinkGeek.com/ ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge
Re: [kbuild-devel] Some feedback on using kbuild
On Tue, 25 Jun 2002 23:06:39 +1000, Greg Banks <[EMAIL PROTECTED]> wrote: >I agree with Keith, shadow trees rock. I think they are probably the single >most useful feature of kbuild 2.5. I fervently hope we end up with shadow >trees or something like them by the end of Linux 2.5. Greg covered almost all the points I was planning to make. One other point: kbuild must not assume that it is running on Linux. Users must be able to build the Linux kernel from _any_ evironment that supports Posix and has a _small_ set of GNU tools. This includes Solaris, Cygwin and other host systems. kbuild must not assume that non-standard tools such as SCM are available. We do not even assume that yacc, lex, loadkeys or Perl are available. These tools may be needed for doing development on certain drivers but kbuild must not require anything beyond gcc and the standard small set of utilities. Bottom line: to support add on code and patch sets kbuild has to do the work itself. --- Sponsored by: ThinkGeek at http://www.ThinkGeek.com/ ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
[CCs trimmed] [Sam Ravnborg] > > Obviously the kernel build system should work for everyone irrespective > > of the SCM system in use. This put at least the following demands: > > 1) Separate OBJ and SRC tree > > 2) That kbuild does not touch any files in the SRC tree Agreed. It looks like Kai is close to that point. Actually VPATH does all the work for you if you let it, though it looks as though Kai (along with kaos) is taking a more manual approach to get more control. (Come to think of it, VPATH can almost do shadow trees - though it would fail to comprehend different layers of Makefiles.) > > The functionality to support shadow trees are best located in an > > SCM tool. If the developer or group in question does not use a > > SCM tool - or maybe a non-coherent setup with two different SCM > > tools then there is a need to support this at another place. > > The best place is in between the kbuild and the trees managed. [Greg Banks] > However in the real world we need to plan for the case where the SCM > is too dumb to do this and too (ahem) precious to be fixed to do > this, or even nonexistant, so we end up having to do it in kbuild. > > This is the same reason why we have the extraordinary complexity of > autoconf and automake for usermode programs: it's regrettably > necessary to deal with imperfect and unfixable platforms. Ah, but note that the Linux kernel build does *not* use autoconf or automake. Why not? Because the platform *is* seen as fixable after all. If your old, weird platform has instead of - tough, you'll have to fix your platform before you compile mkdep.c or split-include.c. If you don't happen to have a perror() - tough again. We assume not only a decent ANSI C compiler - we assume a specific range of gcc versions. Etc. > > It no SCM tool is used then a simple script that takes all updates > > from one or more separate SRC trees and copy them to a common > > place could do it. > > And then afterwards we've got to untangle this mess again, > completely reliably and automatically. The beauty of having proper > shadow trees is that the trees are kept completely separate at all > times except in the compiler. There is *no* danger of ending up > sending the wrong patch to the wrong place or checking the wrong > file into the wrong SCM. There is that. But if you regard "cobble together a build-only source tree" as the first stage of your build process, this shouldn't be a problem. > > > [...] In some (most?) SCM systems, the checkout process on a > > > file resets the timestamp to when it was checked in. > CVS does this. Maybe there's a way to turn it off, I don't know. Huh? Not for me. It sets current time on anything *I* check out. Maybe there's a way to turn that screwy feature *on* > I agree with Keith, shadow trees rock. I think they are probably > the single most useful feature of kbuild 2.5. I'll have to take your word for it - I guess I'm still at Keith's early stages where he was arguing against them before seeing how useful they apparently really are. Peter --- Sponsored by: ThinkGeek at http://www.ThinkGeek.com/ ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
Peter Samuelson wrote: > > [Sam Ravnborg] > > > Obviously the kernel build system should work for everyone irrespective > > > of the SCM system in use. This put at least the following demands: > > > 1) Separate OBJ and SRC tree > > > 2) That kbuild does not touch any files in the SRC tree > > Agreed. It looks like Kai is close to that point. Actually VPATH > does all the work for you if you let it, Yes, VPATH is very useful. > [...] (Come to think of it, VPATH can almost do shadow trees - > though it would fail to comprehend different layers of Makefiles.) I think the problem of Makefile bits in shadow trees is really quite difficult. Keith's solution of pre-processing Makefiles and Makefile.appends from all the shadow trees into a combined Makefile doesn't handle all the cases but is the best attempt I've seen so far. > [Greg Banks] > > This is the same reason why we have the extraordinary complexity of > > autoconf and automake for usermode programs: it's regrettably > > necessary to deal with imperfect and unfixable platforms. > > Ah, but note that the Linux kernel build does *not* use autoconf or > automake. Why not? Because the platform *is* seen as fixable after > all. If your old, weird platform has instead of > - tough, you'll have to fix your platform before you > compile mkdep.c or split-include.c. If you don't happen to have a > perror() - tough again. We assume not only a decent ANSI C compiler - > we assume a specific range of gcc versions. Etc. So now we assume BK? What's next, Python 2.1? > > [...] The beauty of having proper > > shadow trees is that the trees are kept completely separate at all > > times except in the compiler. [...] > > There is that. But if you regard "cobble together a build-only source > tree" as the first stage of your build process, this shouldn't be a > problem. This would be the case if the build process were simple and linear and consisted of just cobbling together a combined source tree and then building a kernel image. But in my experience it comprises a number of loops where I go back and fix simple compile errors (either my own or the latest IDE breakage from the mainline kernel) and do a partial rebuild. A solution where I have to cobble together 174 MB of kernel source every time I fix a one-line compile error is not useful. > > > > [...] In some (most?) SCM systems, the checkout process on a > > > > file resets the timestamp to when it was checked in. > > > CVS does this. Maybe there's a way to turn it off, I don't know. > > Huh? Not for me. It sets current time on anything *I* check out. > Maybe there's a way to turn that screwy feature *on* I guess Mandrake must have done that, because that's how it behaves for me. Here's part of a brand new checkout: gnb@ocelot 1006> ls -l total 584 -rw-r--r--1 gnb sh3 11651 Jun 1 1999 ABOUT-NLS -rw-r--r--1 gnb sh3 13911 Sep 28 2001 CHANGES -rw-r--r--1 gnb sh3 17992 May 29 1999 COPYING -rw-r--r--1 gnb sh394 Jul 25 2001 COPYRIGHT drwxr-xr-x2 gnb sh3 4096 Jun 26 00:29 CVS/ -rw-r--r--1 gnb sh3 5285 Sep 22 2001 Makefile.in -rw-r--r--1 gnb sh3 18670 Sep 28 2001 TODO -rw-r--r--1 gnb sh3 747 Jun 10 1999 argstest.awk -rw-r--r--1 gnb sh3 4208 Jun 10 1999 argstest.dat -rw-r--r--1 gnb sh3 26916 Sep 22 2001 autoconf.c -rw-r--r--1 gnb sh3 2838 Sep 22 2001 autoconf_strs.c -rw-r--r--1 gnb sh3 4008 Sep 21 2001 client.c -rw-r--r--1 gnb sh3 2095 Oct 2 2001 common.h -rwxr-xr-x1 gnb sh3 17285 May 30 1999 config.guess* -rw-r--r--1 gnb sh3 3597 Jul 27 2001 config.h.in -rwxr-xr-x1 gnb sh3 18001 May 30 1999 config.sub* -rw-r--r--1 gnb sh3 4757 Sep 28 2001 configure.in drwxr-xr-x4 gnb sh3 4096 Jun 26 00:29 doc/ -rwxr-xr-x1 gnb sh3 1618 Oct 7 2001 extract_targets.in* -rw-r--r--1 gnb sh3 13057 Sep 2 2001 filter.c -rw-r--r--1 gnb sh3 1619 Jul 25 2001 filter.h -rw-r--r--1 gnb sh3 12961 Sep 22 2001 find.c -rw-r--r--1 gnb sh3 5370 Jul 27 2001 glib_extra.c -rw-r--r--1 gnb sh3 1177 Jul 25 2001 glib_extra.h -rw-r--r--1 gnb sh3 11012 Sep 22 2001 help.c Note the directories are set to current time but the files are set to the time of the last checkin. Greg. -- the price of civilisation today is a courageous willingness to prevail, with force, if necessary, against whatever vicious and uncomprehending enemies try to strike it down. - Roger Sandall, The Age, 28Sep2001. --- Sponsored by: ThinkGeek at http://www.ThinkGeek.com/ ___ kbuild-deve
Re: [kbuild-devel] Some feedback on using kbuild
[Greg Banks] > I think the problem of Makefile bits in shadow trees is really quite > difficult. Keith's solution of pre-processing Makefiles and > Makefile.appends from all the shadow trees into a combined Makefile > doesn't handle all the cases but is the best attempt I've seen so > far. Agreed.. > So now we assume BK? What's next, Python 2.1? Touché. No, my point was not that we can assume BK, but that we can assume the developer is willing to install whatever tools he needs to get the job done. I think the assumption is valid, assuming the developer has some choice in the matter - i.e., do not dictate a specific SCM or even a specific version of gcc (though there is a fairly limited range of acceptible gccs). > This would be the case if the build process were simple and linear > and consisted of just cobbling together a combined source tree and > then building a kernel image. But in my experience it comprises a > number of loops where I go back and fix simple compile errors > (either my own or the latest IDE breakage from the mainline kernel) > and do a partial rebuild. A solution where I have to cobble > together 174 MB of kernel source every time I fix a one-line compile > error is not useful. Another good point. You seem to be full of those. Anyway, the "cobble together" script will most likely build a symlink tree, not a whole separate copy, so you probably wouldn't have to rebuild it *every* compile. At least, if I were writing an ad hoc shadow tree preprocessor, that's how I'd do it. Then when you are just fixing one-liners, you aren't adding or removing files from the tree so you don't rebuild the link forest. > Here's part of a brand new checkout: Ah, I wasn't thinking 'cvs co / export', I was thinking 'cvs update', which is sane. If you have a brand new checkout, you probably don't have a preexisting set of object files yet, so I'm not convinced that you can actually break your compile this way. (I do agree it's a design bug in cvs.) Peter --- Sponsored by: ThinkGeek at http://www.ThinkGeek.com/ ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
On Wed, Jun 26, 2002 at 12:36:11AM +1000, Greg Banks wrote: > I think the problem of Makefile bits in shadow trees is really > quite difficult. Keith's solution of pre-processing Makefiles and > Makefile.appends from all the shadow trees into a combined Makefile > doesn't handle all the cases but is the best attempt I've seen so > far. Keith's approach is the only working attempt to handle shadow trees outside a modern SCM tool. So obviously it is the best so far. > So now we assume BK? What's next, Python 2.1? People are not forced to use BK whatsoever. Last time I looked there were still regulary patches available at kernel.org - where does this come into the picture. But on the other hand, if people want decent support for parrallel development then a modern (not CVS) SCM tools is perferable. Be that BitKeeper, PRCS, arch or whatever. If people refuses to use existing tools solving a specific problem then thay are on their own. I see no point in integrating this deep into kbuild because some people do not want to, or do not understand how to do parrallel development using a proper SCM tool. This does not stop any attemp to make a simple wrapper that creates and maintain a BUILD_TREE. To check timestamps and link accordinly should not take too much time, at least not at the second run. Sam --- Sponsored by: ThinkGeek at http://www.ThinkGeek.com/ ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
Peter Samuelson wrote: > > > So now we assume BK? What's next, Python 2.1? > > Touché. No, my point was not that we can assume BK, but that we can > assume the developer is willing to install whatever tools he needs to > get the job done. > > I think the assumption is valid, assuming the developer has some > choice in the matter - i.e., do not dictate a specific SCM or even a > specific version of gcc (though there is a fairly limited range of > acceptible gccs). I think in theory this is a good rule well stated, but in practice we start to get into some tricky areas. What if the definition of "any SCM tool to get the job done" narrows to the point where there's basically one choice, as with the compiler? Or if it narrows just sufficiently to exclude CVS? I agree we can't support every possible SCM tool in the universe, but there's a lot of impact in the details of where we're willing to draw the line between an SCM "good enough" and one that's not. My point is that if we draw this line such that CVS is on the "good enough" side, we will need to do the shadow tree support outside the SCM, either in kbuild or in your postulated wrapper for kbuild. > [...] Anyway, the > "cobble together" script will most likely build a symlink tree, not a > whole separate copy, so you probably wouldn't have to rebuild it > *every* compile. Interesting. > At least, if I were writing an ad hoc shadow tree preprocessor, that's > how I'd do it. Then when you are just fixing one-liners, you aren't > adding or removing files from the tree so you don't rebuild the link > forest. Well, why don't you write such a thing, so we can compare and contrast with kbuild2.5's approach? I believe that kbuild2.5's approach will stand scrutiny. > > Here's part of a brand new checkout: > > [...]If you have a brand new checkout, you probably don't > have a preexisting set of object files yet, so I'm not convinced that > you can actually break your compile this way. Personally, this doesn't bother me so much, I've never really had the bad luck to trip over some of the truly pathological cases in the kernel. I just wanted to point out that it's not a totally imaginary problem. Greg. -- the price of civilisation today is a courageous willingness to prevail, with force, if necessary, against whatever vicious and uncomprehending enemies try to strike it down. - Roger Sandall, The Age, 28Sep2001. --- Sponsored by: ThinkGeek at http://www.ThinkGeek.com/ ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
On Tue, Jun 25, 2002 at 11:34:09PM +1000, Keith Owens wrote: > > One other point: kbuild must not assume that it is running on Linux. > Users must be able to build the Linux kernel from _any_ evironment that > supports Posix and has a _small_ set of GNU tools. This includes > Solaris, Cygwin and other host systems. kbuild must not assume that > non-standard tools such as SCM are available. We do not even assume > that yacc, lex, loadkeys or Perl are available. These tools may be > needed for doing development on certain drivers but kbuild must not > require anything beyond gcc and the standard small set of utilities. > > Bottom line: to support add on code and patch sets kbuild has to do the > work itself. This does in no way stop you from achiving the same functionality in a small wrapper script. Again, do not clobber kbuild with functionality provided by modern widespread tools. I could come with other similar examples were we rely on other tools but could do it in kbuild instead. Sam --- Sponsored by: ThinkGeek at http://www.ThinkGeek.com/ ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
Sam Ravnborg wrote: > > On Wed, Jun 26, 2002 at 12:36:11AM +1000, Greg Banks wrote: > > I think the problem of Makefile bits in shadow trees is really > > quite difficult. Keith's solution of pre-processing Makefiles and > > Makefile.appends from all the shadow trees into a combined Makefile > > doesn't handle all the cases but is the best attempt I've seen so > > far. > Keith's approach is the only working attempt to handle shadow trees > outside a modern SCM tool. > So obviously it is the best so far. Actually the SuperH port uses a sort of poor man's shadow tree, where we get a shadow tree from CVS and then use a script to symlink the shadow tree's files into the main tree. I understand this approach was copied from another port, although I can't remember which. > > So now we assume BK? What's next, Python 2.1? > People are not forced to use BK whatsoever. Last time I looked there > were still regulary patches available at kernel.org - where does this > come into the picture. Sorry, this was a rhetorical question, I'll try to be clearer next time. > But on the other hand, if people want decent support for parrallel > development then a modern (not CVS) SCM tools is perferable. Be that > BitKeeper, PRCS, arch or whatever. Some people, I believe a significant number, are still stuck in the CVS dark ages. CVS still has some advantages, e.g. you can get a free, public, remote CVS repository at sourceforge.net for basically zero effort. I won't attempt to argue that CVS is better than BK (although I might argue its more practical than PRCS for some tasks). But the reality is that it's still out there, it's mature and it's quirks are well known by those who use it. The reality is also that some people have no SCM at all, our beloved leader being one these until recently. > If people refuses to use existing tools solving a specific problem > then thay are on their own. I see no point in integrating this deep > into kbuild because some people do not want to, or do not understand > how to do parrallel development using a proper SCM tool. The problem is that when you're dealing with a combined tree built from source published by multiple people with varying SCM setups (including none at all) you soon end up working with the lowest common denominator of all those SCM setups, i.e. none at all. Hence, to get all the benefits of (say) BK, we *all* need to be running BK. Until we reach this state of Nirvana, tree management will be very very painful for some people. > This does not stop any attemp to make a simple wrapper that > creates and maintain a BUILD_TREE. > To check timestamps and link accordinly should not take too much time, > at least not at the second run. Ok, why don't you and Peter Samuelson get together, create such a thing and we can compare it against kbuild2.5? If it's simple and a win, great! Greg. -- the price of civilisation today is a courageous willingness to prevail, with force, if necessary, against whatever vicious and uncomprehending enemies try to strike it down. - Roger Sandall, The Age, 28Sep2001. --- This sf.net email is sponsored by: Jabber Inc. Don't miss the IM event of the season | Special offer for OSDN members! JabConf 2002, Aug. 20-22, Keystone, CO http://www.jabberconf.com/osdn ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
On Wed, Jun 26, 2002 at 01:48:04AM +1000, Greg Banks wrote: > Ok, why don't you and Peter Samuelson get together, create such a thing and > we can compare it against kbuild2.5? If it's simple and a win, great! The same reasons as so many other I belive: I do not have the need, I do not have the time, so I do not bother doing it. But I could return the question to you. If you thing this needs to be done by kbuild - why not add it to kbuild. The answer that it is already there in kbuild-2.5 I do not buy into. Reasons for this is simple: - kbuild-2.5 is not in main line - kbuild-2.5 forces a new syntax for makefiles - kbuild-2.5 adds a lot of complexity compared to Rules.make* as of today. * Rules.make needs some commenting to explain the magic behind the constructs. kbuild-2.5 does a good job commenting the src. A big open question for me is still: o What do I get from kbuild-2.5 that justify the added complexity? -> Separate obj and src trees [Soon available in kbuild-2.4] -> Shadow trees -> Faster total build -> Slower build of a single object when no specific syntax is used NO_MAKEFILE_GEN -> Better documentation, and more precise documentation -> Anything else worth to mention? Sam --- This sf.net email is sponsored by: Jabber Inc. Don't miss the IM event of the season | Special offer for OSDN members! JabConf 2002, Aug. 20-22, Keystone, CO http://www.jabberconf.com/osdn ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
[Sam Ravnborg]
> > This does not stop any attemp to make a simple wrapper that
> > creates and maintain a BUILD_TREE.
> > To check timestamps and link accordinly should not take too much
> > time, at least not at the second run.
[Greg Banks]
> Ok, why don't you and Peter Samuelson get together, create such a thing and
> we can compare it against kbuild2.5? If it's simple and a win, great!
Not sure about a win - but the following is certainly simple. It took
me 5 or 10 minutes. Perhaps there are cases it doesn't handle, but
I'm thinking it should work for most things.
TOTALLY UNTESTED, since it's intended solely as a proof of concept.
Also, I don't have any shadow trees already set up.
Peter
#!/bin/sh
# mkshadow: a hack to make a build tree from shadow source trees
#
# If something doesn't like the symlinks, replace 'ln -s' with 'ln'.
# Then you get hard links which should work anywhere, but which easily
# get out of sync with reality when you patch / edit the original
# files, depending on your editor.
if [ -z "$KBUILD_SRCTREE_000" ]; then
echo "no KBUILD_SRCTREE_000, nothing to do"
exit 0
fi
obj=$KBUILD_OBJTREE
test -n "$obj" || obj=.
src=$(set | sed -ne '/^KBUILD_SRCTREE_[0-9][0-9][0-9]=/s//p')
test -n "$src" || src=.
cd $obj
for s in $src; do
if [ ${s#/} = $s ]; then
echo "source tree '$s' is not absolute"
exit 1
elif [ $(cd $s; pwd -P) = $(pwd -P) ]; then
echo "source tree $(pwd -P) is also the object tree"
echo "this is not allowed"
exit 1
fi
(cd $s; find * -type d) | xargs mkdir -p;
(cd $s;
exec find * \( -type d -exec mkdir \{} \; \) -o \
\( -type f ! -name \*.prepend ! -name \*.append -print \) ) |
while read f; do
rm -f $f;
ln -s $s/$f $f
done
done
for s in $src; do
(cd $s;
exec find * -type f -name \*.prepend ) |
while read f; do
base=${f%.prepend}
if [ -f $base ]; then mv $base $base.; else touch $base.; fi
cat $s/$f $base. > $base
rm $base.
done
(cd $s;
exec find * -type f -name \*.append ) |
while read f; do
base=${f%.append}
if [ -f $base ]; then mv $base $base.; else touch $base.; fi
cat $s/$f $base. > $base
rm $base.
done
done
---
This sf.net email is sponsored by: Jabber Inc.
Don't miss the IM event of the season | Special offer for OSDN members!
JabConf 2002, Aug. 20-22, Keystone, CO http://www.jabberconf.com/osdn
___
kbuild-devel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
Sam Ravnborg wrote: >On Tue, Jun 25, 2002 at 11:34:09PM +1000, Keith Owens wrote: > >>One other point: kbuild must not assume that it is running on Linux. >>Users must be able to build the Linux kernel from _any_ evironment that >>supports Posix and has a _small_ set of GNU tools. This includes >>Solaris, Cygwin and other host systems. kbuild must not assume that >>non-standard tools such as SCM are available. We do not even assume >>that yacc, lex, loadkeys or Perl are available. These tools may be >>needed for doing development on certain drivers but kbuild must not >>require anything beyond gcc and the standard small set of utilities. >> >>Bottom line: to support add on code and patch sets kbuild has to do the >>work itself. >> > >This does in no way stop you from achiving the same functionality >in a small wrapper script. >Again, do not clobber kbuild with functionality provided by >modern widespread tools. > I personally think that other tools SHOULD NOT be discounted. They should be considered if: a) they are open source and under a free license such as GPL. b) they are in wide spread use and generally available on every posix system. eg. I don't know of any system that does not run python or perl. They are in wide spread use by most developers and I can't see why they can not be used if deemed fit. c) it is a more efficient way to get the job done. ie. choose the right tool for the right job. It's highly debateable whether GNU Make is the right tool for the build system. It is such and OLD generation of a dependency maintenance tool with OLD design concepts. I think a moderd DMT would really improve a lot of the build issues of the linux kernel (or any project for that matter). I'm not an expert but the one I like to use is "cook". I know of others too (cake, bake, ...). Choosing an inappropriate tool and trying to force it to do things it was not originally designed to do can only lead to more problems (higher maintenace, lack of clarity, obscure hacking, ...). I don't know if kbuild-2.5 fits into this category but I my gut feeling is that some of it probably does. I assume that some of the code that has been written is custom code for linux. Is this necessary ? Surely in and _ideal_ world the tools should be generic enough to apply to any large project. Keith has taken on a huge task and has done an excellent job. His goals are numerous and all of them valid. Well done and I hope this discussion produces some more improvements for the kernel and potentially other projects. Regards, Brendan Simon. --- This sf.net email is sponsored by: Jabber Inc. Don't miss the IM event of the season | Special offer for OSDN members! JabConf 2002, Aug. 20-22, Keystone, CO http://www.jabberconf.com/osdn ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
[I wrote]
> (cd $s; find * -type d) | xargs mkdir -p;
> (cd $s;
>exec find * \( -type d -exec mkdir \{} \; \) -o \
> \( -type f ! -name \*.prepend ! -name \*.append -print \) ) |
> while read f; do
This is redundant - the second mkdir is not needed.
Should be:
(cd $s; exec find * -type d) | xargs mkdir -p;
(cd $s;
exec find * -type f ! -name \*.prepend ! -name \*.append) |
while read f; do
Still untested,
Peter
PS - I still (unlike Sam) like Keith's kbuild2.5. I think it is a
cleaner and more flexible design than the existing makefiles. I just
don't see the need for explicit shadow tree support. (Separate src
and obj dirs are *definitely* needed, though.)
As Kai continues to hack on the existing Rules.make system, I've
changed my mind about what an unfixable mess it was. Obviously some
of its flaws were fixable after all (or some features of kbuild2.5
were mergeable after all, depending on how you look at it). I still
prefer kbuild2.5, but by a significantly smaller margin than I did
before Kai started.
---
This sf.net email is sponsored by: Jabber Inc.
Don't miss the IM event of the season | Special offer for OSDN members!
JabConf 2002, Aug. 20-22, Keystone, CO http://www.jabberconf.com/osdn
___
kbuild-devel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
[Keith Owens]
> prepend/append are logical wrappers, the order is prepend top to
> bottom, base file, append bottom to top. This must be done over all
> source trees.
I think my hack wrapper does that already.
> prepend/append entries in shadow trees below the base version of a
> file must be ignored.
I had that wrong but it is trivially fixable. In fact, doing so
simplifies the script somewhat.
> This script will not automatically cope with files or source trees
> being added or deleted.
That is true. It also doesn't cope with the ${KBUILD_*TREE*}
variables being changed after the initial run.
Which isn't to say you couldn't add such a command to the toplevel
Makefile to be run before every build, similar to what kbuild25 does
in the absense of NO_MAKEFILE_GEN.
> Relying on end users to know when to run manual commands is
> unacceptable, kbuild must automatically detect changes that affect
> the kernel.
End users don't use shadow trees. Developers do. Unless kernel
patches are now being distributed as shadow tarballs instead of patch
files, which AFAICT is not by any means the common case.
Developers should know when to rerun the hypothetical 'make forest'.
If not, it could be automated by $(TOPDIR)/Makefile, see above.
One problem with putting a hack script in $(TOPDIR)/Makefile is that
it causes spurious rebuilds, since it always regenerates from *.*pend
files even when they haven't changed.
> The killer: changing the set of source trees or a file in a source
> tree introduces timestamp skew that make cannot cope with.
This is true. This is a showstopper. From a correctness standpoint,
there is no easy way to solve this problem - if one expects to be able
to add / remove shadow trees "at runtime", one must accept either
running a lengthy (slow) script at that time, or move to kbuild25
where such a thing was explicitly designed for.
> BTW, shadow trees do not "clutter" up kbuild 2.5. The hard part was
> separating source and object and tracking timestamp skew correctly.
Hmmm, depends on perspective. Separate source / object is not hard -
many projects can trivially achieve that with VPATH and a little care.
Of your four sources of timestamp skew, one of these was created by
shadow trees, so it's a problem you had to solve anyway. In the
absense of shadow trees, the other three causes of skew are well known
and with a bit of care can often be ignored. (In fact, ntp can be
used to mitigate nfs, and I believe modern ntpds only skew the clock a
few milliseconds at a time, which is a pretty fast edit / compile /
run cycle if you think about it. As for SCMs, someone claimed that
CVS set times backwards but that is only true on initial checkout,
whereas update is the case where you'd care about this.)
Summary: without shadow trees, the timestamp skew problem almost goes
away anyway, even though theoretically there *are* other sources of
it.
> Going from one directory to two was hard, going from two to many was
> easy.
I'll buy that statement if you mean going from one *source* directory
to two. Remember, though, the feature everyone seems to want is to
have one source directory and one object directory. By all
appearances that really *is* a significantly easier problem.
(Even the -I thing you mention in the next message. Very seldom do
you need access to generated files in custom include directories other
than $PWD.)
Peter
---
This sf.net email is sponsored by: Jabber Inc.
Don't miss the IM event of the season | Special offer for OSDN members!
JabConf 2002, Aug. 20-22, Keystone, CO http://www.jabberconf.com/osdn
___
kbuild-devel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
Sam Ravnborg wrote: > > On Wed, Jun 26, 2002 at 01:48:04AM +1000, Greg Banks wrote: > > Ok, why don't you and Peter Samuelson get together, create such a thing and > > we can compare it against kbuild2.5? If it's simple and a win, great! > The same reasons as so many other I belive: > I do not have the need, I do not have the time, so I do not bother doing it. > > But I could return the question to you. If you thing this needs to be > done by kbuild - why not add it to kbuild. > The answer that it is already there in kbuild-2.5 I do not buy into. Well, until recently that was the entire reason nobody bothered to do anything at all to kbuild, because we all know it would be solved in kbuild-2.5 and that Linus had said he was in favour of kbuild-2.5. Until Linus changed his mind this was a perfectly good plan. > Reasons for this is simple: > - kbuild-2.5 is not in main line Yes. > - kbuild-2.5 forces a new syntax for makefiles Yes. > - kbuild-2.5 adds a lot of complexity compared to Rules.make* as of today. Actually, I would dispute that. Firstly, Rules.make is already quite complex, but the complexity is perhaps not obvious. Secondly, kbuild-2.5 does not replace something simple with something complex, instead it replaces something complex and broken with something of greater complexity and not broken. The complexity added by kbuild-2.5 is added to fix cases which are ignored or poorly handled in kbuild, it's not spurious or bloatful. > * Rules.make needs some commenting to explain the magic behind the > constructs. kbuild-2.5 does a good job commenting the src. I think if Rules.make explained itself as well as Keith's work you would see why the extra complexity is justified. > A big open question for me is still: > o What do I get from kbuild-2.5 that justify the added complexity? > -> Separate obj and src trees [Soon available in kbuild-2.4] > -> Shadow trees > -> Faster total build > -> Slower build of a single object when no specific syntax is used > NO_MAKEFILE_GEN > -> Better documentation, and more precise documentation As Keith keeps pointing out, the build is more accurate. Greg. -- the price of civilisation today is a courageous willingness to prevail, with force, if necessary, against whatever vicious and uncomprehending enemies try to strike it down. - Roger Sandall, The Age, 28Sep2001. --- This sf.net email is sponsored by:ThinkGeek Caffeinated soap. No kidding. http://thinkgeek.com/sf ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Some feedback on using kbuild
Sam Ravnborg wrote: > > On Fri, Jun 28, 2002 at 05:42:53PM +1000, Greg Banks wrote: > > > > [...] kbuild-2.5 does not replace something simple with > > something complex, instead it replaces something complex and broken with > > something of greater complexity and not broken. [...] > > About current kbuild being broken I'm a bit puzzled. The old scheme with > make dep (using mkdep) were indeed broken. But thats fixed now. > Separate SRC and OBJ trees are on the way. > > So whats left in current kbuild that is considered broken? Keith has a file with a long list of kbuild bugs and design flaws, all of which he has fixed in kbuild-2.5. I don't know which of these Kai has addressed, but I'll bet lunch it's a long way short of all of them. > The statement that the builds are more accurate is no longer true. > kbuild tracks ALL dependencies and recompile if the command line changed > as well. > But please prove me wrong I may have overlooked something. I'll let Keith point out all the cases Kai hasn't handled. Greg. -- the price of civilisation today is a courageous willingness to prevail, with force, if necessary, against whatever vicious and uncomprehending enemies try to strike it down. - Roger Sandall, The Age, 28Sep2001. --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
