Re: [ros-dev] Tree restructure (was: Re:[ros-diffs] [akhaldi]48236: Create a branch forcmakebringup.)
From: Ged Murphy Sent: Tuesday, July 27, 2010 12:06 PM To: 'ReactOS Development List' Subject: Re: [ros-dev] Tree restructure (was: Re:[ros-diffs] [akhaldi]48236: Create a branch forcmakebringup.) Timo has given you various scenarios as to where the proposed system is an improvement on the existing. Maybe someone can give an reason as to why the current layout is better than the suggested one? I think it's best to do kind of a summary of pros vs. cons. I remember we spent quite some time discussing it with him, but (Timo - correct me if I'm wrong) his argumentation was that it's more logical, which is quite subjective. So you want more reasons? Agreed about alphabetical, but that's exaggeration. I'd much prefer to go to the audio directory to work on everything from portcls.sys up to mmsys.cpl. Things could be committed together instead of separately forcing 2 separate builds on buildbot Yes, but imagine someone who's going to work on Control Panel Applets, e.g. a UI person (if we had one), who would need to work with all CPLs in the system. He would need to find them throughout whole tree, ask in ros-dev ML if he missed any and abandon his idea soon after starting. Imagine if Linux wasn't modularized and changes made to X or Gnome forced rebuilds of the kernel. It would be carnage. ReactOS is modularized a lot more really. Linux is a ball made of interdependent threads. If they used RBuild, they would get libusb recompiled evertime a comment in the KMail app source code was changed. The proposed layout also lends itself well to the branching system we've been wanting to work towards. Branches could be more easily assigned to specific areas and we'd be able to give out commit access more easily. Agreed about branching. As the tree grows I really don't see how the current layout will remain feasible. Something will have to change and I don't think hacking the build system around a broken tree structure is the answer. What I really want is that if I have four modules: a, b, c and d, which could be located at any relative path inside the build tree. So what I want is being able to specify a few targets, e.g. all being a+b+c+d, kernel being a+b, minwin being a+b+c, and build specified target! So if looking into the future of my example I want minwin to include component d, I don't have to move source code files of the d component, but just mark it as belonging to the minwin group. With your example we would have to reshuffle files everytime we think module's contents should be changed. Also with your example of a new tree layout as an aid for better building, we can't have two targets sharing same modules unless they are physically located under one subdirectory. A substantial constraint really. And overall, it sounds like hacking our tree layout to suit sucky build system, not vice versa. Ged. From: ros-dev-boun...@reactos.org [mailto:ros-dev-boun...@reactos.org] On Behalf Of Aleksey Bragin Sent: 26 July 2010 23:10 To: ReactOS Development List Subject: Re: [ros-dev] Tree restructure (was: Re: [ros-diffs] [akhaldi]48236: Create a branch forcmakebringup.) Please, this would be my last reply to this thread. Yet another time I'm getting an answer that reshuffling files in the directory makes build time shorter. Seriously, am I writing with background and foreground text colors being set to the same value or what? Is there any real, serious reason to break compatibility with all existing branches, make modules harder to find, whatever else, BESIDES hacking around a broken build system which can't have proper grouping? I proposed to properly solve this with either sysgen, cmake or anything else. With a build system which does not suck. Not with a build system, where you need to adjust file paths in order to be able to control build process! I'm glad to participate in a discussion about pros and cons of a proposed new tree layout, but so far the only thing I keep listening to is that it's somehow going to make build time shorter. Let's be honest: It won't. If a 1 liner in PSDK causes whole tree to rebuild, it will take the same with the new layout. It will just be built in a different order, but still all will be rebuilt, because of (somehow broken, or too strict, or incompatible with the makefile) dependencies tracking. It won't make build time shorter until a new build system is in place. WBR, Aleksey Bragin. From: Ged Murphy Sent: Tuesday, July 27, 2010 1:23 AM To: ReactOS Development List Subject: Re: [ros-dev] Tree restructure (was: Re: [ros-diffs] [akhaldi]48236: Create a branch forcmakebringup.) On 26 July 2010 21:29, Aleksey Bragin alek...@reactos.org wrote: Regarding the current layout is logical: We could sort the modules alphabetically, that would be as logical. But it's not reasonable. Great, we came to an agreement: it is logical :). Reasonability
Re: [ros-dev] Tree restructure (was: Re:[ros-diffs] [akhaldi]48236: Create a branch forcmakebringup.)
That made for hard reading. I've split my original points up so anyone following this can understand it. I'll reply when I have some time :) From: Ged Murphy Sent: Tuesday, July 27, 2010 12:06 PM To: 'ReactOS Development List' Subject: Re: [ros-dev] Tree restructure (was: Re:[ros-diffs] [akhaldi]48236: Create a branch forcmakebringup.) Timo has given you various scenarios as to where the proposed system is an improvement on the existing. Maybe someone can give an reason as to why the current layout is better than the suggested one? I think it's best to do kind ofa summary of pros vs. cons. I remember we spent quite some time discussing it with him, but (Timo - correct me if I'm wrong) his argumentation was that it's more logical, which is quite subjective. So you want more reasons? Agreed about alphabetical, but that's exaggeration. Id much prefer to go to the audio directory to work on everything from portcls.sys up to mmsys.cpl. Things could be committed together instead of separately forcing 2 separate builds on buildbot Yes, but imagine someone who's going to work on Control Panel Applets, e.g. a UI person (if we had one), who would need to work with all CPLs in the system. He would need to find them throughout whole tree, ask in ros-dev ML if he missed any and abandon his idea soon after starting. Imagine if Linux wasnt modularized and changes made to X or Gnome forced rebuilds of the kernel. It would be carnage. ReactOS is modularized a lot more really. Linux is a ball made of interdependent threads. If they used RBuild, they would get libusb recompiled evertime a comment in the KMail app source code was changed. The proposed layout also lends itself well to the branching system weve been wanting to work towards. Branches could be more easily assigned to specific areas and wed be able to give out commit access more easily. Agreed about branching. As the tree grows I really dont see how the current layout will remain feasible. Something will have to change and I dont think hacking the build system around a broken tree structure is the answer. What I really want is that if I have four modules: a, b, c and d, which could belocated at any relative path inside the build tree. So what I want is being able to specify a few targets, e.g. all being a+b+c+d, kernel being a+b, minwin being a+b+c, and build specified target! So if looking into the future of my example I want minwin to include component d, I don't have to move source code files of the d component, but just mark it as belonging to the minwin group. With your example we would have to reshuffle files everytime we think module's contents should be changed. Also with your example of a new tree layout as an aid for better building, we can't have two targets sharing same modules unless they are physically located under one subdirectory. A substantial constraint really. And overall, it sounds like hacking our tree layout to suit sucky build system, not vice versa. Ged. From: ros-dev-boun...@reactos.org [mailto:ros-dev-boun...@reactos.org] On Behalf Of Aleksey Bragin Sent: 26 July 2010 23:10 To: ReactOS Development List Subject: Re: [ros-dev] Tree restructure (was: Re: [ros-diffs] [akhaldi]48236: Create a branch forcmakebringup.) Please, this would be my last reply to this thread. Yet another time I'm getting an answer that reshuffling files in the directory makes build time shorter. Seriously, am I writing with background and foreground text colors being set to the same value or what? Is there any real, serious reason to break compatibility with all existing branches, make modules harder to find, whatever else, BESIDES hacking around a broken build system which can't have proper grouping? I proposed to properly solve this with either sysgen, cmake or anything else. With a build system which does not suck. Not with a build system, where you need to adjust file paths in order to be able to control build process! I'm glad to participate in a discussion about pros and cons of a proposed new tree layout, but so far the only thing I keep listening to is that it's somehow going to make build time shorter. Let's be honest: It won't. If a 1 liner in PSDK causes whole tree to rebuild, it will take the same with the new layout. It will just be built in a different order, but still all will be rebuilt, because of (somehow broken, or too strict, or incompatible with the makefile) dependencies tracking. It won't make build time shorter until a new build system is in place. WBR, Aleksey Bragin. From: Ged Murphy Sent: Tuesday, July 27, 2010 1:23 AM To: ReactOS Development List Subject: Re: [ros-dev] Tree restructure (was: Re: [ros-diffs] [akhaldi]48236: Create a branch forcmakebringup.) On 26 July 2010 21:29, Aleksey Bragin alek...@reactos.org wrote: Regarding the current layout is logical: We could sort the modules alphabetically, that would be as logical. But it's not reasonable. Great, we came
Re: [ros-dev] Tree restructure
I’d much prefer to go to the audio directory to work on everything from portcls.sys up to mmsys.cpl. Things could be committed together instead of separately forcing 2 separate builds on buildbot Yes, but imagine someone who's going to work on Control Panel Applets, e.g. a UI person (if we had one), who would need to work with all CPLs in the system. He would need to find them throughout whole tree, ask in ros-dev ML if he missed any and abandon his idea soon after starting. Isn't it possible to do hard links with svn, git or whatever? So as in your example, mmsys.cpl would be located in the audio sub tree, and hardlinked to the UI/cpl directory. All modern filesystems support hard links, that wouldn't suprise me if modern version control systems would do the same. What do you think? ___ Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Re: [ros-dev] Tree restructure
I'd much prefer to go to the audio directory to work on everything from portcls.sys up to mmsys.cpl. Things could be committed together instead of separately forcing 2 separate builds on buildbot Yes, but imagine someone who's going to work on Control Panel Applets, e.g. a UI person (if we had one), who would need to work with all CPLs in the system. He would need to find them throughout whole tree, ask in ros-dev ML if he missed any and abandon his idea soon after starting. I can't see how hard that would be to do. Build ROS, do a search for .cpl files in output-i386, make a note of their paths then find their equivalent source files. Andrew Faulds http://ajf.me/ ___ Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Re: [ros-dev] Tree restructure (was:Re:[ros-diffs] [akhaldi]48236: Create a branch forcmakebringup.)
Sorry, for some reason Windows Mail screwed up when I was answering between the lines in your email. ( That's why I like MacOS X's mail client :) ) Thanks for reposting with quotation marks. WBR, Aleksey.. -- From: Ged Murphy gedmur...@gmail.com Sent: Wednesday, July 28, 2010 6:45 PM To: 'ReactOS Development List' ros-dev@reactos.org Subject: Re: [ros-dev] Tree restructure (was:Re:[ros-diffs] [akhaldi]48236:Createa branch forcmakebringup.) That made for hard reading. I've split my original points up so anyone following this can understand it. ___ Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Re: [ros-dev] Tree restructure
And here is what I said. http://svnbook.red-bean.com/en/1.0/ch07s03.html Jérôme Gardou wrote: I’d much prefer to go to the audio directory to work on everything from portcls.sys up to mmsys.cpl. Things could be committed together instead of separately forcing 2 separate builds on buildbot Yes, but imagine someone who's going to work on Control Panel Applets, e.g. a UI person (if we had one), who would need to work with all CPLs in the system. He would need to find them throughout whole tree, ask in ros-dev ML if he missed any and abandon his idea soon after starting. Isn't it possible to do hard links with svn, git or whatever? So as in your example, mmsys.cpl would be located in the audio sub tree, and hardlinked to the UI/cpl directory. All modern filesystems support hard links, that wouldn't suprise me if modern version control systems would do the same. What do you think? ___ Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev ___ Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Re: [ros-dev] Tree restructure (was: Re: [ros-diffs] [akhaldi]48236: Create a branch forcmakebringup.)
Timo has given you various scenarios as to where the proposed system is an improvement on the existing. Maybe someone can give an reason as to why the current layout is better than the suggested one? So you want more reasons? Instead of focusing on the build, which is where most of our arguments have lay so far, I'll discuss the merits from a tree structure perspective: Firstly, I don't agree with the 'easier to find' statement. This may be true for someone with no knowledge of NT, but reactos devs are supposed to know this stuff. If the 'easier to find' statement was valid, then I think Timo's suggestion of laying everything out alphabetically would be better still. I'd much prefer to go to the audio directory to work on everything from portcls.sys up to mmsys.cpl. Things could be committed together instead of separately forcing 2 separate builds on buildbot You have to look to the future. Let's consider how Microsoft will do things. Do you think the Windows source code is dumped into one big TFS database with everything filed under module type? This would mean Joe on the kernel team would be effected by what Sam in the audio department is doing. Changes made by Steve on the COM team might force a rebuild of what Ian is working on in the shell team. You can't expect everyone to have a separate branch of the entire source code to avoid this problem. The code is modularized and people are given access to the areas which they work on. Imagine if Linux wasn't modularized and changes made to X or Gnome forced rebuilds of the kernel. It would be carnage. The proposed layout also lends itself well to the branching system we've been wanting to work towards. Branches could be more easily assigned to specific areas and we'd be able to give out commit access more easily. - Student devs could be given access to the applications directory. - We could build up an ntcore directory, make it bootable (aka minwin) and kernel devs could be given branches to this area without the noise of a full OS. - Graphics guys could be checkout the build tools, an ntcore and a win32core areas. They could have write access to win32core and work on that without worrying about what the student devs are doing in the apps dir. As the tree grows I really don't see how the current layout will remain feasible. Something will have to change and I don't think hacking the build system around a broken tree structure is the answer. Ged. From: ros-dev-boun...@reactos.org [mailto:ros-dev-boun...@reactos.org] On Behalf Of Aleksey Bragin Sent: 26 July 2010 23:10 To: ReactOS Development List Subject: Re: [ros-dev] Tree restructure (was: Re: [ros-diffs] [akhaldi]48236: Create a branch forcmakebringup.) Please, this would be my last reply to this thread. Yet another time I'm getting an answer that reshuffling files in the directory makes build time shorter. Seriously, am I writing with background and foreground text colors being set to the same value or what? Is there any real, serious reason to break compatibility with all existing branches, make modules harder to find, whatever else, BESIDES hacking around a broken build system which can't have proper grouping? I proposed to properly solve this with either sysgen, cmake or anything else. With a build system which does not suck. Not with a build system, where you need to adjust file paths in order to be able to control build process! I'm glad to participate in a discussion about pros and cons of a proposed new tree layout, but so far the only thing I keep listening to is that it's somehow going to make build time shorter. Let's be honest: It won't. If a 1 liner in PSDK causes whole tree to rebuild, it will take the same with the new layout. It will just be built in a different order, but still all will be rebuilt, because of (somehow broken, or too strict, or incompatible with the makefile) dependencies tracking. It won't make build time shorter until a new build system is in place. WBR, Aleksey Bragin. From: Ged mailto:gedmur...@gmail.com Murphy Sent: Tuesday, July 27, 2010 1:23 AM To: ReactOS mailto:ros-dev@reactos.org Development List Subject: Re: [ros-dev] Tree restructure (was: Re: [ros-diffs] [akhaldi]48236: Create a branch forcmakebringup.) On 26 July 2010 21:29, Aleksey Bragin alek...@reactos.org wrote: Regarding the current layout is logical: We could sort the modules alphabetically, that would be as logical. But it's not reasonable. Great, we came to an agreement: it is logical :). Reasonability is discussable... I'm yet to hear any arguments as to how the current layout is better than the suggested one. As the tree grows in size it's going to become more and more difficult to manage. Do we really have to wait until we're at a point where it takes 5 hours to build after making a 1 line change to a PSDK file? As far as I can tell, our current layout
Re: [ros-dev] Tree restructure (was: Re: [ros-diffs] [akhaldi] 48236: Create a branch forcmakebringup.)
On 26 July 2010 21:29, Aleksey Bragin alek...@reactos.org wrote: Regarding the current layout is logical: We could sort the modules alphabetically, that would be as logical. But it's not reasonable. Great, we came to an agreement: it is logical :). Reasonability is discussable... I'm yet to hear any arguments as to how the current layout is better than the suggested one. As the tree grows in size it's going to become more and more difficult to manage. Do we really have to wait until we're at a point where it takes 5 hours to build after making a 1 line change to a PSDK file? As far as I can tell, our current layout, by type, only serves to make modules easy to find. In comparison, Timo's alphabetical point is actually as reasonable as the current layout. Ged. ___ Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Re: [ros-dev] Tree restructure (was: Re: [ros-diffs] [akhaldi]48236: Create a branch forcmakebringup.)
Please, this would be my last reply to this thread. Yet another time I'm getting an answer that reshuffling files in the directory makes build time shorter. Seriously, am I writing with background and foreground text colors being set to the same value or what? Is there any real, serious reason to break compatibility with all existing branches, make modules harder to find, whatever else, BESIDES hacking around a broken build system which can't have proper grouping? I proposed to properly solve this with either sysgen, cmake or anything else. With a build system which does not suck. Not with a build system, where you need to adjust file paths in order to be able to control build process! I'm glad to participate in a discussion about pros and cons of a proposed new tree layout, but so far the only thing I keep listening to is that it's somehow going to make build time shorter. Let's be honest: It won't. If a 1 liner in PSDK causes whole tree to rebuild, it will take the same with the new layout. It will just be built in a different order, but still all will be rebuilt, because of (somehow broken, or too strict, or incompatible with the makefile) dependencies tracking. It won't make build time shorter until a new build system is in place. WBR, Aleksey Bragin. From: Ged Murphy Sent: Tuesday, July 27, 2010 1:23 AM To: ReactOS Development List Subject: Re: [ros-dev] Tree restructure (was: Re: [ros-diffs] [akhaldi]48236: Create a branch forcmakebringup.) On 26 July 2010 21:29, Aleksey Bragin alek...@reactos.org wrote: Regarding the current layout is logical: We could sort the modules alphabetically, that would be as logical. But it's not reasonable. Great, we came to an agreement: it is logical :). Reasonability is discussable... I'm yet to hear any arguments as to how the current layout is better than the suggested one. As the tree grows in size it's going to become more and more difficult to manage. Do we really have to wait until we're at a point where it takes 5 hours to build after making a 1 line change to a PSDK file? As far as I can tell, our current layout, by type, only serves to make modules easy to find. In comparison, Timo's alphabetical point is actually as reasonable as the current layout. Ged. ___ Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev___ Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Re: [ros-dev] Tree restructure
Aleksey Bragin wrote: Please, this would be my last reply to this thread. Yet another time I'm getting an answer that reshuffling files in the directory makes build time shorter. Seriously, am I writing with background and foreground text colors being set to the same value or what? Is there any real, serious reason to break /compatibility with all existing branches,/ This is a valid point. One of the few to take care of. make modules harder to find, It will actually make modules much easier to find. When I look for a module I usually know what kind of module it is. When I look for gdi32, I know it's a core win32 thing and I don't have to look through hundreds of wine dlls, just to find gdi32. If I don't know what kind of module that is, I should either stay away from it, or simply use doxygen, that will tell me where it is. But this should be really the exception. whatever else, BESIDES hacking around a broken build system which can't have proper grouping? Did I write in chineese letters on greek background? How can the build system care for shorter build times if we always do a clean rebuild of *everything*? It can't. How can a different file layout do that? Easy: the buildbot can decide what part of reactos was commited to and only clean and rebuild THAT part. If I commit some audio file, just rebuild the audio part. The rest of the tree is unaffected. We also don't recompile gcc when someone commited to the kernel. The only dependency that all stuff shares is the SDK, containing the core headers and core libs like crt. This way we could divide our buildbot's build time probably by 5 to 10. Now please tell me how rbuild or cmake or sysgen can do that? Another thing is dependencies. Currently everyone just adds dependencies wherever he likes. There's so much interdependency you can't just not build something like dlls. For the amd64 port, I added a bunch of hacks to only compile kernel mode stuff to save a lot of compilation time, but that took me quite some time to get it working and it's pretty hacky. The new file layout would allow to simply comment out one directory from the root folder and still everything would build. I proposed to properly solve this with either sysgen, cmake or anything else. With a build system which does not suck. Not with a build system, where you need to adjust file paths in order to be able to control build process! I'm glad to participate in a discussion about pros and cons of a proposed new tree layout, but so far the only thing I keep listening to is that it's somehow going to make build time shorter. The only thing on the other side of the argumentation that I heard so far is, that it doesn't matter how bad our file layout is, we can work around it with a complex build system. But that's not true. Let's be honest: It won't. If a 1 liner in PSDK causes whole tree to rebuild, it will take the same with the new layout. It will just be built in a different order, but still all will be rebuilt, because of (somehow broken, or too strict, or incompatible with the makefile) dependencies tracking. It won't make build time shorter until a new build system is in place. WBR, Aleksey Bragin. ___ Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Re: [ros-dev] Tree restructure
Just a reminder to all involved in this discussion, the work with cmake has one goal, which is to remove our dependency on rbuild and the maintenance load that using it entails. rbuild has a lot of issues by itself, irrespective of the problems it causes in combination with the directory hierarchy, but trying to go to cmake isn't designed to solve those problems, just the ones rbuild itself causes. It may ultimately make solving the combined issues easier, but that's a secondary, albeit important, consideration. That restructuring of the directories will likely be necessary has very little to do with the cmake effort itself. So let's not conflate the two issues. ___ Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
