Re: [PATCH 1/1] Adapt on_each_cpu
Am Montag 11 August 2008 02:53:44 schrieb Stephane Marchesin: On 8/2/08, Jerome Glisse [EMAIL PROTECTED] wrote: I might be totaly wrong so feel free to ignore these. I got the feeling that the user test base on linux kernel is far bigger than ours. Also i think that our test user base are people wanting lastest things with old kernel, while i understand that (building kernel is not fun on my ram slim computer) i think this end up being a burden to us. So in the end i think we should be better off with linux development tree where dev know the deadline to get feature in. I got the feeling that this way we could drive development on features basis like getting vblank rework done for a given kernel release and so get dev to focus more on some features and get them done in a timely fashion. This way we could avoid to get some new feature to rot a bit in the drm tree because. Also i think the linux-next or other linux bleeding edge tree would give us lot more tester with a lot more experience on good bug report that our current test base (i am not saying that we have bad tester, we have some very good one too which we should give credits, just that we might be able to get more of them). Judging by the current trend (where we see lots of people reporting the recent shmem_file_setup breakage because they tried to load git drm on a non-tip kernel), we have a lot of testers that don't run latest kernels but still get drm git. So the argument of more testers may not be true. As another data point, I belong to the drm git on distribution-provided kernel crowd but I would be happy to switch to the proposed in-kernel tree system. Here's why. As far as I can tell, there are three possible setups I could have: 1) The status quo (i.e. out-of-kernel drm tree) where I only compile the DRM module against my standard, distribution-provided kernel. 2) The status quo, but I compile both a recent kernel and the DRM module from the out-of-kernel drm tree. 3) The proposed new system where DRM development is done in the kernel tree. Now to be honest, of all these options I do find 1) to be the simplest option, but that simply *fails* when the DRM depends on advanced kernel developments. This leaves option 2) and 3) and quite frankly, I *really* don't like option 2) because it forces me to keep track of multiple trees that are not explicitly coupled. This is a recipe for build failures - after all, which kernel tree is guaranteed to always have all the things that the latest DRM tree needs? Is it Linus' tree? -mm? Something else entirely? As long as the DRM depends on recent kernel developments, option 3) is really the most friendly way to go forward. There's only one tree to track for me and I can simply use distribution-provided tools for the build. Of course, I'm probably not really representative, so take all this with a grain of salt. cu, Nicolai - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ -- ___ Dri-devel mailing list Dri-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dri-devel
Re: [PATCH 1/1] Adapt on_each_cpu
On 8/2/08, Jerome Glisse [EMAIL PROTECTED] wrote: I might be totaly wrong so feel free to ignore these. I got the feeling that the user test base on linux kernel is far bigger than ours. Also i think that our test user base are people wanting lastest things with old kernel, while i understand that (building kernel is not fun on my ram slim computer) i think this end up being a burden to us. So in the end i think we should be better off with linux development tree where dev know the deadline to get feature in. I got the feeling that this way we could drive development on features basis like getting vblank rework done for a given kernel release and so get dev to focus more on some features and get them done in a timely fashion. This way we could avoid to get some new feature to rot a bit in the drm tree because. Also i think the linux-next or other linux bleeding edge tree would give us lot more tester with a lot more experience on good bug report that our current test base (i am not saying that we have bad tester, we have some very good one too which we should give credits, just that we might be able to get more of them). Judging by the current trend (where we see lots of people reporting the recent shmem_file_setup breakage because they tried to load git drm on a non-tip kernel), we have a lot of testers that don't run latest kernels but still get drm git. So the argument of more testers may not be true. Now that I think about it, is there a way with git to : 1. have a single main drm branch (that is, keep drm git the way it is right now) 2. inside this branch, maintain a number of changesets (each of those changesets would be an in-development feature). It seems to me we'd get the best of both worlds that way; the changesets would let us submit features upstream easily, and no push/pull would be needed to update all the repos. Seeing how most drm developments do not overlap, requiring explicit pull/pushes with merges sounds more complex than it needs to be. Now I'm not sure if it's possible, but at least I don't see a technical reason this wouldn't. Stephane - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ -- ___ Dri-devel mailing list Dri-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dri-devel
Re: [PATCH 1/1] Adapt on_each_cpu
On Thu, Aug 07, 2008 at 04:08:49PM -0700, Eric Anholt wrote: a) BSD I'd like to hear Robert's concerns here, but I've been working with some of the BSD folks lately, and it seems like the main concerns are: 1) making it easy for contributors to identify which portions of code are shared 2) licensing Since both the BSDs and Linux effectively copy code out of the DRM repo and into their respective kernel trees at this point, having the actual repo based on one of them shouldn't be an issue as long as both 1 and 2 are handled. The remaining compat macros could probably just be wrapped in some sort of Linux equivalent (DRM_SPINUNLOCK-spin_unlock, what's the difference?), or we could annotate things for the BSD guys to run scripts over. As it stands, they still have to go over things by hand anyway... As an ex-BSD kernel maintainer, I stood against moving to a linux kernel-based tree for a long time. For a long time I felt like I was the only guy holding back the move. I couldn't get NetBSD to work in the upstream tree, and it sounds like OpenBSD's following the same route, so maybe I was doing it wrong all along in trying to have one tree for all OSes to share. As the OpenBSD maintainer it's probably time I mention my reasons for working out of tree. It's quite simple really, In my experience of porting over the code, I found that the BSDs, while similar, are no where near identical, and in the end the level of #ifdefing in the bsd-core area just got insane, it made maintainance a nightmare. So I started working out of tree. If i find a general bug, I reformat against git and send the patch upstream. Otherwise, I watch what happens to git and merge it into my kernel tree along with any OpenBSD specific changes i've needed. This is more like the way the BSDs have shared code for a while now. I know myself and Robert differ on our opinions here, though. I find it better to be able to go over things by hand, it means I better understand what I'm merging, anyway. For that reason, i'm not against master going to a linux kernel tree, since it would change my process very minimally. As long as the licensing doesn't change (IMHO drm is essentially a part of X, and thus should be X11/MIT licensed), I'm fine with it. Process-wise, I'm in agreement with Dave Airlie, that master should track what's in linus' tree, with the rest on branches. Also, I think vblank-rework is now stable enough to go upstream, i've found it has started to work quite nicely. Just my 2p, -0- -- There's no easy quick way out, we're gonna have to live through our whole lives, win, lose, or draw. -- Walt Kelly - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ -- ___ Dri-devel mailing list Dri-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dri-devel
Re: [PATCH 1/1] Adapt on_each_cpu
On Thu, 2008-07-31 at 19:53 -0700, Jesse Barnes wrote: On Thursday, July 31, 2008 6:40 pm Dave Airlie wrote: Well, if the overhead of merging upstream is a concern, then how about not worrying about bc at all and let people who want to back port deal with it? Oh, and what about just keeping the drm drivers in a linux kernel tree? That'll make upstream merging even easier yet... The less crap I have to remove from the diffs the better, I have a script and it removes stuff mostly fine, maybe someone should actually bring this idea up on the list with a proper plan for how to deal with a) BSD I'd like to hear Robert's concerns here, but I've been working with some of the BSD folks lately, and it seems like the main concerns are: 1) making it easy for contributors to identify which portions of code are shared 2) licensing Since both the BSDs and Linux effectively copy code out of the DRM repo and into their respective kernel trees at this point, having the actual repo based on one of them shouldn't be an issue as long as both 1 and 2 are handled. The remaining compat macros could probably just be wrapped in some sort of Linux equivalent (DRM_SPINUNLOCK-spin_unlock, what's the difference?), or we could annotate things for the BSD guys to run scripts over. As it stands, they still have to go over things by hand anyway... As an ex-BSD kernel maintainer, I stood against moving to a linux kernel-based tree for a long time. For a long time I felt like I was the only guy holding back the move. I couldn't get NetBSD to work in the upstream tree, and it sounds like OpenBSD's following the same route, so maybe I was doing it wrong all along in trying to have one tree for all OSes to share. Entries 1 and 2 in the list were of concern to me for quite some time, as those championing the move to a linux tree were doing it primarily for GPL and style motivations (at least vocally). I didn't like the GPL motivation, and felt like we had other avenues to fix the style motivation (which I eventually worked on, and was about half done with I think). The other problem as a BSD maintainer was how to merge code. Originally all I had available was CVS and cp, so the shared-code system we had was useful -- I just worked in the DRM tree and periodically copied changes over and cvs committed. Eventually I got perforce access, and was trying to use that to manage merges, but CVS IDs and perforce diff formatting and performance issues made it more work than copying diffs by hand. I settled on a series of seds on the DRM tree and cping it to the BSD tree, which worked reasonably well. Luckily, those times are past. We've got git now. If I was doing BSD at this point, I would probably rather set it up as a branch of a linux-2.6 kernel tree that I merged from: Initial setup today: cd linux-2.6 git-checkout -b freebsd-drm cd drivers/gpu/drm cp ~/src/drm/bsd-core/drm*.[ch] . cp ~/src/drm/bsd-core/i915/Makefile i915/ cp ~/src/drm/bsd-core/i915*.[ch] i915/ (etc) git-add . git-commit -a -v Next merges: git-merge origin/master (resolve conflicts and make it compile again as linux-specific bits trickle in) git commit -a Then I could use my old script for copying an external tree into the FreeBSD repo and continue doing mega-commits as before. The advantage here is that since I'm merging the linux-specific OS side, I don't have to go and read the two side-by-side all the time to make sure they haven't drifted too far apart, which was almost all of the time I spent working on the BSD code. This is assuming that the two codebases stay close enough together for the merger to do something sane, but I think that would be the case for the most part. To improve that, I'd add more compat macros to leave the FreeBSD side more linux-looking. FreeBSD's grown a layer for taking osol code mostly as-is for the ZFS and dtrace ports, and I think growing some compatibility for taking mostly linux-looking code would probably be a sensible plan for maintaining this codebase, and would probably help various other potential FreeBSD projects as well. I don't think anyone is proposing changing the licensing of DRM code this time around, so with this plan as a BSD guy I wouldn't even care which parts of the code were shared versus os-specific. b) backwards compat for using updated DRM drivers on older kernels. Since I'm now maintaining a linux-2.6 tree for getting my GEM work merged, I'm having to deal with backporting concerns already. The plan I'm proposing for our project is for us to do topic branches like other linux kernel developers, propose them for merge either through you or directly, and maintain one branch for all the things I want on top of current linux kernel to get what we need. As we head towards our quarterly release, I'll prepare a branch against the latest released kernel of everything we need for the quarterly release, which is then the stuff that backporters would
Re: [PATCH 1/1] Adapt on_each_cpu
On Fri, Aug 1, 2008 at 6:27 PM, Dave Airlie [EMAIL PROTECTED] wrote: Personally, I only use the existing DRM repo on old kernels because that's how it's structured. It's actually more work for me to download build a recent kernel, then update build the DRM drivers against it that it is to simply update the DRM drivers and build against my current kernel (or just updating a theoretical DRM kernel tree building for that matter). So really I don't think keeping compat *in-tree* for old kernels gives us extra testing. It's really no harder or easier to do the same thing with a full kernel tree, just different. IMO anyway. Yeah, we could keep the BC code around if we moved the drivers in-tree, and that would let you compile the drivers against an older kernel using something like: First you said this :).. Another benefit from having the DRM repo structured as linux kernel tree is that changes from upstream linux development (janitorial stuff, tree-wide api changes) only takes a 'git merge' to carry over to the DRM tree. In other words, making it easier to push changes upstream works both ways, it also becomes easier to pull down changes from upstream that touches the DRM subystem. Hm, I guess that's what your saying in 4). Then this, the thing is to keep it building you need compat code, code that can't go into Linus tree, so we end up with a tree that isn't like Linus tree, and we still have to patch manage transitions so we don't save anything doing this over what we have now. I was thinking that having the BC code in-tree might be acceptable. As far as I remember, the old firewire stack did that for a while, but maybe they've tightened up the policy on that. So, I very much agree with your proposal and don't feel I can add much, except to point out that a migration to in-tree drm development doesn't need to be a big and painful process. Basically, we just decide to do it, and designate a kernel tree on freedesktop.org that we work from and start with the current state of upstream drm. There will be some work in moving things over from drm.git to the kernel tree, but we were going to do that *anyway* as part of getting it upstream. What will be different is that Dave won't have to do all that work, we can split it up between us and all Dave will need to do is to merge the branch and push it to Linus. This is of course what Eric is already doing with GEM in people.fd.o/~anholt/linux-2.6, and that would be a great way to get the vblank rework upstream too. If we do this, I'm going to introduce new rules for features/patches that will put a lot more work on people. 1. Nobody gets commit access to master branch except me. 2. All work is done in topic branches that are throwaway once complete. So when the branch owner decides the work is upstreamable, They create a set of clean patches against the current master, send them to dri-devel for review, if they pass review then the branch owner creates a new clean branch and asks me to merge it, it also includes Signed-off-by lines. Some people say we want the history, Linus wants bisectable history so invariable when a feature reaches a useful stage it should go upstream in clean s-o-b patches that are bisectable and pass checkpatch.pl, not in a 50 separate patches with fixes in it. We can generate a throwaway drm-next tree that merges all the topic branches currently in play if people want to have some superview. 3. All API changes to go to the list as soon as possible (should be done now). That all sounds good to me, I think we will benefit from a little stricter process instead of the free-for-all drm.git repo we have now. And topic branches sounds very promising; if GEM had been done in a topic branch off of the upstream linux tree, preparing the patch would literally be one git diff command, instead of having to sort out the GEM changes from drm.git. Similarly, Ian's XGI work wouldn't have used the ttm fences and gotten upstream faster. Kristian - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ -- ___ Dri-devel mailing list Dri-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dri-devel
Re: [PATCH 1/1] Adapt on_each_cpu
As we discussed on IRC last night, I think these changes are perfectly reasonable (in fact just what I'd expect if we moved to this model). Sure, it will force contributors to be more disciplined, but that's probably a good thing anyway. I'd still like to hear from the BSD guys about how we can best keep shareable code obvious and contribution conditions clear, but so far I haven't heard any serious problems. Well, I'm only like a third of a developer, but I'll give it a shot. You are correct that the most important thing for me is being able to readily identify what code is or should be portable and licensed appropriately. I suppose in many ways this is easier for oga to swallow because he already works in his own tree. I however work entirely out of drm.git and just drop that into our cvs/svn tree for releases. If we move to a linux tree, does that mean that I will have to pull a full linux git tree and then try to hand pick the drm bits out? Yeah, that's what I'd expect. Where would the bsd specific code live? I guess it would be in the apprpriate BSD CVS or something? What about libdrm? That should probably be in its own repo. Do I have to setup my own repo for the bsd parts and just try to merge stuff from the linux tree? Yeah, although given what Dave mentioned, you'd also see patches proposed submitted more visibly on dri-devel before going into the repo. A BSD DRM maintainer could actually setup a separate, BSD specific repo along the lines of the Linux one, and merge patches just like Dave will be doing for Linux. That might be easiest (and who knows, maybe even better than what we have currently from a BSD perspective). As for BC, as the code stands now, the bsd side of the house has a few ifdefs that enable certain features from certain releases of bsd, but it pretty much can just drop into any 6,7,8 release as-is. That may become more of a pain as I try and rework various code paths taking advantage of newer features, but for now it mostly just works. Sounds like the BSD camp moves its internal APIs a bit more slowly than Linux. Certainly makes things easier for driver people... I realize that some of the new features KMS and GEM are relying much more heavily on specific kernel features, which already makes my task daunting. I don't think that anyone is intentionally trying to make things more difficult, but I don't see how this move doesn't make it even more difficult for me to try and keep pace with linux crowd. I know I need help already and if I have to keep up with every patch and bugfix that goes in and cherry-pick it, I won't have time to do much else. If anything, I think this will actually slow down the pace of commits on the Linux side. We should see fewer, higher quality commits than under the current scheme, so with a BSD gatekeeper setup like I mentioned above things might even get easier... Thanks, Jesse - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ -- ___ Dri-devel mailing list Dri-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dri-devel
Re: [PATCH 1/1] Adapt on_each_cpu
Then this, the thing is to keep it building you need compat code, code that can't go into Linus tree, so we end up with a tree that isn't like Linus tree, and we still have to patch manage transitions so we don't save anything doing this over what we have now. I was thinking that having the BC code in-tree might be acceptable. As far as I remember, the old firewire stack did that for a while, but maybe they've tightened up the policy on that. Not allowed anymore, including version.h is a red flag nowadays.. stricter process instead of the free-for-all drm.git repo we have now. And topic branches sounds very promising; if GEM had been done in a topic branch off of the upstream linux tree, preparing the patch would literally be one git diff command, instead of having to sort out the GEM changes from drm.git. Similarly, Ian's XGI work wouldn't have used the ttm fences and gotten upstream faster. Well XGI isn't going upstream as last I heard from IBM it still didn't work, and they scrapped the XGI plan and bought a bunch of X1650s once Ben and myself fixed the endianness issues. While I'd like to believe preparing the patch would be one git diff away, I suspect it would have been 15 cherry-picks, a couple of merges, a few interactive rebases and a batch of checkpatch.pls away. Nothing is ever a git diff away from upstream. So we would need to find a way to get all the compat hacks into two/three files with no version checks in the actual code. Sounds icky. Or someone who cared enough would need to maintain a separate repo for old kernels. Dave. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ -- ___ Dri-devel mailing list Dri-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dri-devel
Re: [PATCH 1/1] Adapt on_each_cpu
On Fri, 1 Aug 2008 17:29:39 +0100 Keith Whitwell [EMAIL PROTECTED] wrote: The biggest thing I have against this is that it cuts our testing base down, we have a small testing base already, cutting it further jsut to benefit some developers who are frustrated isn't really a gain. I don't have a particular attachment to the standalone drm tree concept. It really does date from another era. Back then we had different ideas about how people would get their drivers and what OS's would be enthusiastic enough to want to join the party. It is nice to be able to build new drm against older kernels, however -- I'd be a bit sad to lose that possibility. If that can somehow be preserved, at least for relatively recent kernels (perhaps a 1 year window??), then there's nothing I've got against this. I might be totaly wrong so feel free to ignore these. I got the feeling that the user test base on linux kernel is far bigger than ours. Also i think that our test user base are people wanting lastest things with old kernel, while i understand that (building kernel is not fun on my ram slim computer) i think this end up being a burden to us. So in the end i think we should be better off with linux development tree where dev know the deadline to get feature in. I got the feeling that this way we could drive development on features basis like getting vblank rework done for a given kernel release and so get dev to focus more on some features and get them done in a timely fashion. This way we could avoid to get some new feature to rot a bit in the drm tree because. Also i think the linux-next or other linux bleeding edge tree would give us lot more tester with a lot more experience on good bug report that our current test base (i am not saying that we have bad tester, we have some very good one too which we should give credits, just that we might be able to get more of them). Thought, i might be wrong, just my feelings here. Cheers, Jerome Glisse [EMAIL PROTECTED] - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ -- ___ Dri-devel mailing list Dri-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dri-devel
Re: [PATCH 1/1] Adapt on_each_cpu
On Thu, Jul 31, 2008 at 10:53 PM, Jesse Barnes [EMAIL PROTECTED] wrote: On Thursday, July 31, 2008 6:40 pm Dave Airlie wrote: Well, if the overhead of merging upstream is a concern, then how about not worrying about bc at all and let people who want to back port deal with it? Oh, and what about just keeping the drm drivers in a linux kernel tree? That'll make upstream merging even easier yet... The less crap I have to remove from the diffs the better, I have a script and it removes stuff mostly fine, maybe someone should actually bring this idea up on the list with a proper plan for how to deal with a) BSD Jesse, thanks for writing this up more coherently and politely than I could ever do. It's easy for me to say that's what I had in mind now that you've written it all down, that is pretty much the case. I'd like to hear Robert's concerns here, but I've been working with some of the BSD folks lately, and it seems like the main concerns are: 1) making it easy for contributors to identify which portions of code are shared 2) licensing Since both the BSDs and Linux effectively copy code out of the DRM repo and into their respective kernel trees at this point, having the actual repo based on one of them shouldn't be an issue as long as both 1 and 2 are handled. The remaining compat macros could probably just be wrapped in some sort of Linux equivalent (DRM_SPINUNLOCK-spin_unlock, what's the difference?), or we could annotate things for the BSD guys to run scripts over. As it stands, they still have to go over things by hand anyway... I think both of the above can be addressed fairly easily no matter which tree we choose (obviously I'd like to see a Linux kernel based tree :), by appropriately commenting shared files and making sure contributors understand the licensing implications of their contributions (maybe a drivers/gpu specific README.licensing or somesuch). Agree on all this. b) backwards compat for using updated DRM drivers on older kernels. Personally, I only use the existing DRM repo on old kernels because that's how it's structured. It's actually more work for me to download build a recent kernel, then update build the DRM drivers against it that it is to simply update the DRM drivers and build against my current kernel (or just updating a theoretical DRM kernel tree building for that matter). So really I don't think keeping compat *in-tree* for old kernels gives us extra testing. It's really no harder or easier to do the same thing with a full kernel tree, just different. IMO anyway. Yeah, we could keep the BC code around if we moved the drivers in-tree, and that would let you compile the drivers against an older kernel using something like: make -C /lib/modules/$(uname -r)/build M=$PWD from the drivers/gpu/drm directory. If enough DRI developers are willing to take the stand that they don't care about a or b, then I'm willing to change the development model. So far I've hear grumbles and mumbling, where I expect coherent proposal or gtfo. The biggest thing I have against this is that it cuts our testing base down, we have a small testing base already, cutting it further jsut to benefit some developers who are frustrated isn't really a gain. Testing is part of the problem with the current set up. What are people actually testing when they run drm.git drivers with their distro kernel? Right now drm.git has the vblank bits, the ttm bits and various smaller stuff. Much of this stuff interacts in non-trivial ways, such as the intel irq handler, and hand picking changes, editing the patch to apply upstream, pretty much invalidates any testing done on the drm.git repo. So the closer the drm repo can be to upstream, the more effective the testing will be. To be fair, this isn't all because of the out-of-tree drm.git setup. We've failed, as a community, to get a lot of stuff upstream (vblank stuff took a long time to get ready, there's the memory manager shoot out etc), and as a result, the gap between drm.git and upstream has grown hard to manage. Well, I think there are other potential gains. What are the major problems with the current scheme, maybe we can address those without affecting the issues you mentioned above? In my view, the big issues are: 1) we don't really know what's going upstream and when (there's not a standard Linux s-o-b process for DRM) until it happens 2) DRM changelogs are basically useless since they get destroyed when going upstream, which means many of our commit messages are unhelpful, to say the least (look at most of modesetting-101 for lolz) 3) having a different development process like this increases the barrier to entry for potential developers (this is mostly speculative since we don't know how many potential developers we have, but I think it's still a valid point) 4) Linux moves fast and I think having an out-of-tree project means we're
Re: [PATCH 1/1] Adapt on_each_cpu
Personally, I only use the existing DRM repo on old kernels because that's how it's structured. It's actually more work for me to download build a recent kernel, then update build the DRM drivers against it that it is to simply update the DRM drivers and build against my current kernel (or just updating a theoretical DRM kernel tree building for that matter). So really I don't think keeping compat *in-tree* for old kernels gives us extra testing. It's really no harder or easier to do the same thing with a full kernel tree, just different. IMO anyway. Yeah, we could keep the BC code around if we moved the drivers in-tree, and that would let you compile the drivers against an older kernel using something like: First you said this :).. Another benefit from having the DRM repo structured as linux kernel tree is that changes from upstream linux development (janitorial stuff, tree-wide api changes) only takes a 'git merge' to carry over to the DRM tree. In other words, making it easier to push changes upstream works both ways, it also becomes easier to pull down changes from upstream that touches the DRM subystem. Hm, I guess that's what your saying in 4). Then this, the thing is to keep it building you need compat code, code that can't go into Linus tree, so we end up with a tree that isn't like Linus tree, and we still have to patch manage transitions so we don't save anything doing this over what we have now. So, I very much agree with your proposal and don't feel I can add much, except to point out that a migration to in-tree drm development doesn't need to be a big and painful process. Basically, we just decide to do it, and designate a kernel tree on freedesktop.org that we work from and start with the current state of upstream drm. There will be some work in moving things over from drm.git to the kernel tree, but we were going to do that *anyway* as part of getting it upstream. What will be different is that Dave won't have to do all that work, we can split it up between us and all Dave will need to do is to merge the branch and push it to Linus. This is of course what Eric is already doing with GEM in people.fd.o/~anholt/linux-2.6, and that would be a great way to get the vblank rework upstream too. If we do this, I'm going to introduce new rules for features/patches that will put a lot more work on people. 1. Nobody gets commit access to master branch except me. 2. All work is done in topic branches that are throwaway once complete. So when the branch owner decides the work is upstreamable, They create a set of clean patches against the current master, send them to dri-devel for review, if they pass review then the branch owner creates a new clean branch and asks me to merge it, it also includes Signed-off-by lines. Some people say we want the history, Linus wants bisectable history so invariable when a feature reaches a useful stage it should go upstream in clean s-o-b patches that are bisectable and pass checkpatch.pl, not in a 50 separate patches with fixes in it. We can generate a throwaway drm-next tree that merges all the topic branches currently in play if people want to have some superview. 3. All API changes to go to the list as soon as possible (should be done now). I'm sure I'll think of some more. Dave. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ -- ___ Dri-devel mailing list Dri-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dri-devel
Re: [PATCH 1/1] Adapt on_each_cpu
On Fri, 2008-08-01 at 15:45 -0700, Jesse Barnes wrote: On Friday, August 1, 2008 3:27:33 pm Dave Airlie wrote: So, I very much agree with your proposal and don't feel I can add much, except to point out that a migration to in-tree drm development doesn't need to be a big and painful process. Basically, we just decide to do it, and designate a kernel tree on freedesktop.org that we work from and start with the current state of upstream drm. There will be some work in moving things over from drm.git to the kernel tree, but we were going to do that *anyway* as part of getting it upstream. What will be different is that Dave won't have to do all that work, we can split it up between us and all Dave will need to do is to merge the branch and push it to Linus. This is of course what Eric is already doing with GEM in people.fd.o/~anholt/linux-2.6, and that would be a great way to get the vblank rework upstream too. If we do this, I'm going to introduce new rules for features/patches that will put a lot more work on people. 1. Nobody gets commit access to master branch except me. 2. All work is done in topic branches that are throwaway once complete. So when the branch owner decides the work is upstreamable, They create a set of clean patches against the current master, send them to dri-devel for review, if they pass review then the branch owner creates a new clean branch and asks me to merge it, it also includes Signed-off-by lines. Some people say we want the history, Linus wants bisectable history so invariable when a feature reaches a useful stage it should go upstream in clean s-o-b patches that are bisectable and pass checkpatch.pl, not in a 50 separate patches with fixes in it. We can generate a throwaway drm-next tree that merges all the topic branches currently in play if people want to have some superview. 3. All API changes to go to the list as soon as possible (should be done now). I'm sure I'll think of some more. As we discussed on IRC last night, I think these changes are perfectly reasonable (in fact just what I'd expect if we moved to this model). Sure, it will force contributors to be more disciplined, but that's probably a good thing anyway. I'd still like to hear from the BSD guys about how we can best keep shareable code obvious and contribution conditions clear, but so far I haven't heard any serious problems. Well, I'm only like a third of a developer, but I'll give it a shot. You are correct that the most important thing for me is being able to readily identify what code is or should be portable and licensed appropriately. I suppose in many ways this is easier for oga to swallow because he already works in his own tree. I however work entirely out of drm.git and just drop that into our cvs/svn tree for releases. If we move to a linux tree, does that mean that I will have to pull a full linux git tree and then try to hand pick the drm bits out? Where would the bsd specific code live? What about libdrm? Do I have to setup my own repo for the bsd parts and just try to merge stuff from the linux tree? I want us to be able to move as quickly as we can to deliver new features and functionality, but I see this move as just another step towards making drm even more linux-centric. As for BC, as the code stands now, the bsd side of the house has a few ifdefs that enable certain features from certain releases of bsd, but it pretty much can just drop into any 6,7,8 release as-is. That may become more of a pain as I try and rework various code paths taking advantage of newer features, but for now it mostly just works. I realize that some of the new features KMS and GEM are relying much more heavily on specific kernel features, which already makes my task daunting. I don't think that anyone is intentionally trying to make things more difficult, but I don't see how this move doesn't make it even more difficult for me to try and keep pace with linux crowd. I know I need help already and if I have to keep up with every patch and bugfix that goes in and cherry-pick it, I won't have time to do much else. robert. If you're still nervous about making the move (I would be), we could consider doing a trial run of the new development model for 2.6.28, but we'd probably want to start right away if 2.6.28 really is our target. Jesse - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ -- ___ Dri-devel mailing list Dri-devel@lists.sourceforge.net
Re: [PATCH 1/1] Adapt on_each_cpu
On Wed, Jul 30, 2008 at 10:58 PM, Dave Airlie [EMAIL PROTECTED] wrote: What do you think? We try to keep #ifdef's out of the code and in drm_compat.h instead. Something like #if linuxversion = 2.6.27 #define drm_on_each_cpu(handler, data, wait) ... #else ... #endif and then just user drm_on_each_cpu in the code. I'd prefer not to do that at all, it makes upstream merging a lot messier. The ifdefs make life easier as I have a script to remove them :) Well, if the overhead of merging upstream is a concern, then how about not worrying about bc at all and let people who want to back port deal with it? Oh, and what about just keeping the drm drivers in a linux kernel tree? That'll make upstream merging even easier yet... Kristian - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ -- ___ Dri-devel mailing list Dri-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dri-devel
Re: [PATCH 1/1] Adapt on_each_cpu
Well, if the overhead of merging upstream is a concern, then how about not worrying about bc at all and let people who want to back port deal with it? Oh, and what about just keeping the drm drivers in a linux kernel tree? That'll make upstream merging even easier yet... The less crap I have to remove from the diffs the better, I have a script and it removes stuff mostly fine, maybe someone should actually bring this idea up on the list with a proper plan for how to deal with a) BSD b) backwards compat for using updated DRM drivers on older kernels. If someone writes a coherent proposal I'm all ears, all I'm hearing is other kernel subsystems, but on my review 2 other kernel subsystem have a major userspace part to worry about, alsa and v4l. They both developed in out-of-tree with backwards compat code. If enough DRI developers are willing to take the stand that they don't care about a or b, then I'm willing to change the development model. So far I've hear grumbles and mumbling, where I expect coherent proposal or gtfo. The biggest thing I have against this is that it cuts our testing base down, we have a small testing base already, cutting it further jsut to benefit some developers who are frustrated isn't really a gain. Dave. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ -- ___ Dri-devel mailing list Dri-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dri-devel
Re: [PATCH 1/1] Adapt on_each_cpu
On Thursday, July 31, 2008 6:40 pm Dave Airlie wrote: Well, if the overhead of merging upstream is a concern, then how about not worrying about bc at all and let people who want to back port deal with it? Oh, and what about just keeping the drm drivers in a linux kernel tree? That'll make upstream merging even easier yet... The less crap I have to remove from the diffs the better, I have a script and it removes stuff mostly fine, maybe someone should actually bring this idea up on the list with a proper plan for how to deal with a) BSD I'd like to hear Robert's concerns here, but I've been working with some of the BSD folks lately, and it seems like the main concerns are: 1) making it easy for contributors to identify which portions of code are shared 2) licensing Since both the BSDs and Linux effectively copy code out of the DRM repo and into their respective kernel trees at this point, having the actual repo based on one of them shouldn't be an issue as long as both 1 and 2 are handled. The remaining compat macros could probably just be wrapped in some sort of Linux equivalent (DRM_SPINUNLOCK-spin_unlock, what's the difference?), or we could annotate things for the BSD guys to run scripts over. As it stands, they still have to go over things by hand anyway... I think both of the above can be addressed fairly easily no matter which tree we choose (obviously I'd like to see a Linux kernel based tree :), by appropriately commenting shared files and making sure contributors understand the licensing implications of their contributions (maybe a drivers/gpu specific README.licensing or somesuch). b) backwards compat for using updated DRM drivers on older kernels. Personally, I only use the existing DRM repo on old kernels because that's how it's structured. It's actually more work for me to download build a recent kernel, then update build the DRM drivers against it that it is to simply update the DRM drivers and build against my current kernel (or just updating a theoretical DRM kernel tree building for that matter). So really I don't think keeping compat *in-tree* for old kernels gives us extra testing. It's really no harder or easier to do the same thing with a full kernel tree, just different. IMO anyway. And if there *were* a regular, Linux-style DRM tree, it would be easier to pull in changes from other subsystem trees for testing purposes (e.g. the PCI read base stuff, or the shmem stuff assuming Al has a tree somewhere with it, or the suspend/resume API stuff or what have you). If someone writes a coherent proposal I'm all ears, all I'm hearing is other kernel subsystems, but on my review 2 other kernel subsystem have a major userspace part to worry about, alsa and v4l. They both developed in out-of-tree with backwards compat code. Unfortunately, I don't think either of those code bases do things particularly well either. Upstream changes are often invisible to actual users for much longer than they should be, and the conflation of the userspace code with the external repo means that compatibility is actually *harder* to maintain, not easier, since interfaces aren't separated enough to force people to think about them. Or in the case of v4l, things just progress really slowly until the next rewrite. To be fair, I think we haven't been horrible about this in the DRM (yet). Generally old stuff still works on new kernels, which is important. But we also don't have a userspace component nearly as large as the ALSA guys for example; the number of ioctls we have is still pretty manageable. If enough DRI developers are willing to take the stand that they don't care about a or b, then I'm willing to change the development model. So far I've hear grumbles and mumbling, where I expect coherent proposal or gtfo. The biggest thing I have against this is that it cuts our testing base down, we have a small testing base already, cutting it further jsut to benefit some developers who are frustrated isn't really a gain. Well, I think there are other potential gains. What are the major problems with the current scheme, maybe we can address those without affecting the issues you mentioned above? In my view, the big issues are: 1) we don't really know what's going upstream and when (there's not a standard Linux s-o-b process for DRM) until it happens 2) DRM changelogs are basically useless since they get destroyed when going upstream, which means many of our commit messages are unhelpful, to say the least (look at most of modesetting-101 for lolz) 3) having a different development process like this increases the barrier to entry for potential developers (this is mostly speculative since we don't know how many potential developers we have, but I think it's still a valid point) 4) Linux moves fast and I think having an out-of-tree project means we're often out of date, see the recent
Re: [PATCH 1/1] Adapt on_each_cpu
On Wed, Jul 30, 2008 at 8:19 AM, Johannes Engel [EMAIL PROTECTED] wrote: Hi folks, this rather trivial patch makes drm compile again on kernel = 2.6.27. It is necessary since in kernel 2.6.27 on_each_cpu (defined in include/linux/smp.h) lost the third argument (retry). What do you think? We try to keep #ifdef's out of the code and in drm_compat.h instead. Something like #if linuxversion = 2.6.27 #define drm_on_each_cpu(handler, data, wait) ... #else ... #endif and then just user drm_on_each_cpu in the code. thanks, Kristian - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ -- ___ Dri-devel mailing list Dri-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dri-devel
Re: [PATCH 1/1] Adapt on_each_cpu
What do you think? We try to keep #ifdef's out of the code and in drm_compat.h instead. Something like #if linuxversion = 2.6.27 #define drm_on_each_cpu(handler, data, wait) ... #else ... #endif and then just user drm_on_each_cpu in the code. I'd prefer not to do that at all, it makes upstream merging a lot messier. The ifdefs make life easier as I have a script to remove them :) Dave. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ -- ___ Dri-devel mailing list Dri-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dri-devel
