Re: Discussion of guidelines for additional version control mechanisms (fwd)
> Certainly -- the intent that I expressed in my original e-mail was to fish > for people's thoughts on the issue, which would then be codified in some > form. I'm interested in hearing a little more back on how people feel > about the notion of how larger projects should coordinate work given the > assumption that "locks" are advisory, but plan to write up a strawman > sometime in the next week or so. Probably the process forward there will > be to run the first pass by the core team for some immediate feedback > regarding the strength of the recommendations, and whether it meets our > goals concerning addressing the problems that have been discussed. Then > I'll spit a copy out for more general comments. Sounds like a great plan to me. Alas I think you'll get more feedback once you have the strawman proposal, people just seem to work that way. Later, George -- George V. Neville-Neil [EMAIL PROTECTED] NIC:GN82 "Those who would trade liberty for temporary security deserve neither" - Benjamin Franklin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
On Thu, 28 Feb 2002, George V. Neville-Neil wrote: > > What I mean by "imposing structure" is: > > > > - Identify patterns of development and structure that seem to have evolved > > naturally as part of the maturing of the FreeBSD Project > > - Determine which patterns tend to result in the most productive and > > parallel development efforts, not to mention which are the most popular > > - Selectively reinforce those patterns to improve the ability of > > developers to rely on the patterns > > This sounds fine to me but we're going to have to write it down > somewhere and then discuss it. This tossing ideas in the ether is not > going to work because to refer back to earlier points we have to either > have huge emails with everyone's comments in them, or constantly be > reading the archives. > > So, please, if you think this is important write down your > findings/current beliefs somewhere (I think TWiki or something like it > is perfect for this) and let everyone comment and modify them until we > have an understanding. Then lets figure out what structure we need and > write that down as well. Think of it as a FreeBSD constitution if you > must, only much more lightweight. > > You're obviously taking the lead here and that's fine, every project > (and this discussion is really a project) needs a leader. Certainly -- the intent that I expressed in my original e-mail was to fish for people's thoughts on the issue, which would then be codified in some form. I'm interested in hearing a little more back on how people feel about the notion of how larger projects should coordinate work given the assumption that "locks" are advisory, but plan to write up a strawman sometime in the next week or so. Probably the process forward there will be to run the first pass by the core team for some immediate feedback regarding the strength of the recommendations, and whether it meets our goals concerning addressing the problems that have been discussed. Then I'll spit a copy out for more general comments. > Yes, you read me correctly. My mantra is, "Write it down." Something I've had in mind all along. BTW, your writings are much appreciated -- I hope to make the developer summit notes available in the next couple of days (I'm currently at an Apple-hosted security conference through Friday, when I'll get a chance to sit down with it over the weekend). Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
> What I mean by "imposing structure" is: > > - Identify patterns of development and structure that seem to have evolved > naturally as part of the maturing of the FreeBSD Project > - Determine which patterns tend to result in the most productive and > parallel development efforts, not to mention which are the most popular > - Selectively reinforce those patterns to improve the ability of > developers to rely on the patterns This sounds fine to me but we're going to have to write it down somewhere and then discuss it. This tossing ideas in the ether is not going to work because to refer back to earlier points we have to either have huge emails with everyone's comments in them, or constantly be reading the archives. So, please, if you think this is important write down your findings/current beliefs somewhere (I think TWiki or something like it is perfect for this) and let everyone comment and modify them until we have an understanding. Then lets figure out what structure we need and write that down as well. Think of it as a FreeBSD constitution if you must, only much more lightweight. You're obviously taking the lead here and that's fine, every project (and this discussion is really a project) needs a leader. > I think we're in the same boat here. You believe that process can help > streamline the development process, and be used to help avoid the > disagreements we've run into recently (assuming I read you correctly :-). > I think so too. I agree we can't bring too much process--that won't work > for a large number of reasons. But a little bit of process can go a long > way. In particular, I'm looking for a little bit of process that will > help address the perceived problems of the current situation. > Yes, you read me correctly. My mantra is, "Write it down." :-) Later, George -- George V. Neville-Neil [EMAIL PROTECTED] NIC:GN82 "Those who would trade liberty for temporary security deserve neither" - Benjamin Franklin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
On Wed, 27 Feb 2002, George V. Neville-Neil wrote: > I think we need to avoid the concept of "imposing some modicum of > structure." If we create structure it is because we need it. Just like > software. There was a good comment recently about "software gets > created to scratch an itch." I'd say that structure gets created > because you're tired of losing fingers and toes to the person next to > you wielding the axe. It's great that your friends are helping you > clear the forest but you'd all like to be able to walk and pick up your > cup of coffee at the end of the day as well. What I mean by "imposing structure" is: - Identify patterns of development and structure that seem to have evolved naturally as part of the maturing of the FreeBSD Project - Determine which patterns tend to result in the most productive and parallel development efforts, not to mention which are the most popular - Selectively reinforce those patterns to improve the ability of developers to rely on the patterns An example of this might be investment of time in a rote API change: such changes are time-consuming requiring a lot of brute force source code modification. There's a strong motiviation to avoid redundant work here, but frequently the work is necessary, so it's also desirable to know that it's going to happen. A normal model here might be to look for a developer who's willing to do it, and wait for them to come back in a week or two when it's done. An unproductive tendancy is for four developers to go off, and do the work over two weeks, and all discover they've done it at the end. This can be avoided by making an attempt to coordinate. On a small scale and for individual cases, the "problem" here can be lived with. When it's in the context of a larger piece of work, it can be highly counter-productive. One reinforcement that could be made is to encourage developers to indicate what works in progress they have going, and to encourage other developers interested in a change to give that list a skim to see if someone else has already started on it, and if someone has but appears to be timing out, to encourage them to talk to the developer who has claimed it (in some sense) to find out what is going on. > We are always going to have "first past the post" problems. If someone > comes along and has rewritten a whole subsystem, and testing and > performance measurement show that it's an order of magnitude better > (faster, smaller, etc.) than what we have we'd be idiots not to take it, > right? Obviously. But the contentious cases often aren't the ones where that's the case. > The question is "What processes do we need to put in place to make a > project of this size and dynamisticity work?" I put forward a few of > them. I suggest we start somewhere (including airing gripes people have > with the current system) and write down (i.e. build a web page, use > TWiki) what we're going to do about it. > > I hate to make this analogy but we need a constitution or something like > it. Not so grandiose of course, but a written set of rules and are easy > to interpret that can take care of the 80% case. The 20% we'll always > get to argue over but I'd rather not argue over the 100%. I think we're in the same boat here. You believe that process can help streamline the development process, and be used to help avoid the disagreements we've run into recently (assuming I read you correctly :-). I think so too. I agree we can't bring too much process--that won't work for a large number of reasons. But a little bit of process can go a long way. In particular, I'm looking for a little bit of process that will help address the perceived problems of the current situation. Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
> One of the disagreements that seems to be evolving is whether or not the > project formally supports a task-oriented structure. A couple of people > have asserted that people might claim tasks (such as myself) and by virtue > of claiming the task, be provided with some notion of ownership that is > supported in a more formal sense. Others have pointed out that in a > volunteer environment, people simply do what they want to regardless of > any task ownership, and would prefer a first-past-the-post model to a task > ownership model. My assumption had been that disagreement existed to some > extent based on the nature and strength of ownership, but it seems that > I've made a fundamental assumption there that not everyone agrees with. > > My feeling has always been that imposing some modicrum of structure is > important: to avoid people stepping on toes, people can announce what > they're working on, and expect that others might avoid replicating the > work, or at least be communicated with before it happens. The rationale > for this lies both in efficiency (non-replication of work), and to avoid > toe-stepping, since there's a natural notion of ownership over work done, > and a desire to not see it discarded. Perhaps this can't be supported in > our environment. I think we need to avoid the concept of "imposing some modicum of structure." If we create structure it is because we need it. Just like software. There was a good comment recently about "software gets created to scratch an itch." I'd say that structure gets created because you're tired of losing fingers and toes to the person next to you wielding the axe. It's great that your friends are helping you clear the forest but you'd all like to be able to walk and pick up your cup of coffee at the end of the day as well. We are always going to have "first past the post" problems. If someone comes along and has rewritten a whole subsystem, and testing and performance measurement show that it's an order of magnitude better (faster, smaller, etc.) than what we have we'd be idiots not to take it, right? The question is "What processes do we need to put in place to make a project of this size and dynamisticity work?" I put forward a few of them. I suggest we start somewhere (including airing gripes people have with the current system) and write down (i.e. build a web page, use TWiki) what we're going to do about it. I hate to make this analogy but we need a constitution or something like it. Not so grandiose of course, but a written set of rules and are easy to interpret that can take care of the 80% case. The 20% we'll always get to argue over but I'd rather not argue over the 100%. Later, George -- George V. Neville-Neil [EMAIL PROTECTED] NIC:GN82 "Those who would trade liberty for temporary security deserve neither" - Benjamin Franklin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
+---[ George V. Neville-Neil ]-- | > In addition to process it might be attitude. | > | | So, how do we get our attitudes adjusted before hitting a wall, | as many companies I've worked for did? Alcohol and a cam-corder d8) -- Totally Holistic Enterprises Internet| | Andrew Milton The Internet (Aust) Pty Ltd | | ACN: 082 081 472 ABN: 83 082 081 472 | M:+61 416 022 411 | Carpe Daemon PO Box 837 Indooroopilly QLD 4068|[EMAIL PROTECTED]| To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
> In addition to process it might be attitude. > It might be a stretch but they are kind of the same. Good processes come from good attitude. It is extraordinarily hard for most people (especially smart people) to be introspective on such questions when they think they already have the answer. Sometimes people think that because something works for them it will work in a group. But we know that's not always the case. So, how do we get our attitudes adjusted before hitting a wall, as many companies I've worked for did? It comes back to agreeing on a process by which we work. We have one now, it may not all be written down, but we do have it. Later, George -- George V. Neville-Neil [EMAIL PROTECTED] NIC:GN82 "Those who would trade liberty for temporary security deserve neither" - Benjamin Franklin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
> There are only only 8 core team members, unless you mean something > different by "core" here than [EMAIL PROTECTED] I guess I was going based on the meeting I attended back at BSD Con. > Otherwise, I tend to agree with your basic premise that it is process > and not tools at the heart of this problem. Of that I am glad. Later, George -- George V. Neville-Neil [EMAIL PROTECTED] NIC:GN82 "Those who would trade liberty for temporary security deserve neither" - Benjamin Franklin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
: My feeling has always been that imposing some modicrum of structure is : important: to avoid people stepping on toes, people can announce what : they're working on, and expect that others might avoid replicating the : work, or at least be communicated with before it happens. The rationale : for this lies both in efficiency (non-replication of work), and to avoid : toe-stepping, since there's a natural notion of ownership over work done, : and a desire to not see it discarded. Perhaps this can't be supported in : our environment. In the past, we haven't been strictly a "first one past the post" project. We have rejected things as not being ready for inclusion in FreeBSD all the time. Jon Chen's Cardbus work was the second or third effort that was presented to FreeBSD. It was the first one "acceptable" to those folks that took a look at it. Sure, it had its problems, but it was something that could be worked with and people generally agreed that it was the direction we wanted to go. The first one that was presented worked as well, but there were some issues with the direction that it took, so we didn't integrate it. We had the first CardBus implementation almost 9 months or a year before the second one. Of course, looking at the past to get precedent can be dicy, since we've done many things right as a project and many things wrong. :-) Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
One of the disagreements that seems to be evolving is whether or not the project formally supports a task-oriented structure. A couple of people have asserted that people might claim tasks (such as myself) and by virtue of claiming the task, be provided with some notion of ownership that is supported in a more formal sense. Others have pointed out that in a volunteer environment, people simply do what they want to regardless of any task ownership, and would prefer a first-past-the-post model to a task ownership model. My assumption had been that disagreement existed to some extent based on the nature and strength of ownership, but it seems that I've made a fundamental assumption there that not everyone agrees with. My feeling has always been that imposing some modicrum of structure is important: to avoid people stepping on toes, people can announce what they're working on, and expect that others might avoid replicating the work, or at least be communicated with before it happens. The rationale for this lies both in efficiency (non-replication of work), and to avoid toe-stepping, since there's a natural notion of ownership over work done, and a desire to not see it discarded. Perhaps this can't be supported in our environment. Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
In message <[EMAIL PROTECTED]>, "M. Warner Losh" writes: >In message: <[EMAIL PROTECTED]> >"George V. Neville-Neil" <[EMAIL PROTECTED]> writes: >: The problem here is process. The FreeBSD project now has more than >: 12 core members and more than 12 committers. With any number larger >: than 12 it is VERY HARD to reach consensus on anything. Without a >: process by which we agree to reach consensus it is impossible. > >There are only only 8 core team members, unless you mean something >different by "core" here than [EMAIL PROTECTED] > >Otherwise, I tend to agree with your basic premise that it is process >and not tools at the heart of this problem. In addition to process it might be attitude. Core@ was not elected to be ornamental. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
Hi Folks, Before I address Robert's questions directly I'd like to make a few comments. Now, I'm not a committer, I'm a newbie as far as working on FreeBSD itself goes, but I've been a faithful consumer of this stuff since early NetBSD days and was runing Free around 2.2.7. I am also a software engineer with a lot of experience in dealing with OS development and various development models. I spent several years of my time at Wind River being called into meetings on just these subjects, so I do have a bit of a clue and I'm sure that many people out there have similar information but I'd like to put it down in green and black (well whatever colors you've picked...) The problem is NOT (as David Greenman has pointed out) a particular tool. I've used CVS and Clear Case. I have not used Perforce but I know some folks who worked on it and I have a clue as to how it works as well. I find that I like CVS's simplicity but I understand why people could want to use Clear Case as well and I have heard good things about Perforce. In this case I do NOT have a bias about the tools. The problem here is process. The FreeBSD project now has more than 12 core members and more than 12 committers. With any number larger than 12 it is VERY HARD to reach consensus on anything. Without a process by which we agree to reach consensus it is impossible. I read with some amusement the discussion of locks that Robert's email touched off. That discussion showed the group's confusion on this point. Optimally the "tree" would only be locked for such reasons as labelling it, or during a code freeze. Locking sections of it so someone can work on it does not make sense in a distributed project. We'll spend all our time in email doing human locking protocols and will probably lead to more code forking. This is bad. What would a possible solution look like? 1) We need to decide as a group how the framework that supports the OS is broken up and how it is glued together. This is the biggest technical problem, we need to make it so that pervasive changes are not necessary and so that APIs between chunks are either versioned or as static as possible. 2) Once we have areas an area lead will volunteer (probably for a period of time not exceeding a year and not less than 3 months) for each section and be vetted by the core team. 3) The area lead's job is to coordinate all work to be integrated into his/her section on some sort of schedule. We have a 3 times per year release on -STABLE that's a good driver IMHO. We need to do something like that with -CURRENT and perhaps a snapshot schedule should be set up. 4) All locking and release policies come from the above rules as well as the core team and area leads. The problem we are trying to tackle is partly human. How do we coordinate work in a volunteer project and produce a high quality product? It can either be done with a small team, or with fascist techniques (though this does not work well with volunteers) or by lots of cooperation. Now, to answer Robert's questions. > Question 1: How should the presence of on-going work in an external > repository be announced to the broader community? > > Suggestion: e-mail to -arch, -current, or a related mailing list > Email is fine. A pointer to a web page with design docs etc. should also be part of that. > Question 2: How should the status of on-going work be announced to the > broader community? > > Suggestion: Bi-monthly developer status report > Suggestion: Status web page for the project Collapse the two, put that status report on the web (in TWiki?) > Suggestion: Regular status reports on work to relevant mailing lists > Just have a cron job send out the web links in email. > Question 3: How should the results of the on-going work be made available > to the broader community? > > Suggestion: cvsup10 export of the Perforce tree > Suggestion: patchsets sent to appropriate mailing lists with status > Suggestion: patchsets generated automatically and posted to the mailing > list Patches can really suck. You want one main line of development and only have a few small branches for projects. > Question 4: How agressively should on-going work be pushed back into the > base tree? (Note that the answer here depends a great deal on the nature > of the work: both its rationale, its goals, etc). In particular note that > currently Perforce is used to hold experimental changes, as well as ones > destined for eventual production use. > > Suggestion: For work requiring large source tree sweeps, API changes, etc, > only when the work is ready to commit. API changes are most dangerous. The API change should be announced well in advance of committing it so that people who depend on the API can get a clue before they get screwed. > Suggestion: For more minor work, P4 might be used only for brief > collaboration, or to ass
Re: Discussion of guidelines for additional version control mechanisms(fwd)
On Tue, 26 Feb 2002, Robert Watson wrote: > > On Tue, 26 Feb 2002, Julian Elischer wrote: > > > > On the other hand, you could easily argue that the expectations might be > > > much lower for smaller pieces of work. For example, the move to td_ucred > > > required a substantial amount of infrastructure, but the patches > > > themselves are relatively sane and small. Once the pieces are all in > > > place (which they mostly are), knowing that someone has a lock on it is > > > probably worth a "are you still working on this" followed by a wait of a > > > week or two to see if it turns up before forging ahead. > > > > Bad choice.. > > p->p_ucred => td->td_ucred in istself requires almost no > > infrastracture change, the bulk of the commit is trivial, (AND STILL NOT > > COMMITTED) and waiting for weeks on such a trivial thing before being able > > to proceed on some interlocking piece is foolish. > > I was referring to its dependency on KSE. > > > > (1) The timeout begins when contention occurs, of the lock has been > > > declared. This means that if you seriously intend to do some work, > > > you can say "I'm going to do the work", but you don't risk losing the > > > lock until someone comes to you and says "Hey did you ever...". > > > > Locks shouldn't be unilateral and they shouldn't last more that the > > amount of time that it takes to import a change and get it stable. i.e. > > maybe a week or so at most. Someone used KSE-II as an example.. They > > said I had a 2 month lock (??) I don't know where they got that idea > > from.. I had a vague concensus that maybe things may be broken for a DAY > > OR TWO. while the commit was happenning. I the end it was about right. > > So it sounds like we disagree on what a lock is. My interpretation of the > word lock was that it refered to ownership of a task. For example, we > gave you ownership of the general task of pushing KSE forwards. This > means that if someone turns up and says "I have SAs, I did it entirely > differently, I'm going to commit it now" we might say "Hold just a second > there". We'd probably say that even if they said "I have SAs, I did them > entirely the same way, I'm going to commit it now". ] Sore point: phk: "I've got DEVFS, I did it differently it's much better than tho old fully working one. I'm going to check it in now" core: "OK" phk: Ok I deleted parts of the old devfs so that it's now totally uselss. [2 years pass, system has no useable devfs..] phk: ok here's my devfs.. of course it doesn't work as well as the old one but it's much better. [checks in non working code] oops: spends 3 months getting bugs ironed out. Completed devfs still doesn't do (by design) as much as the old one.. > > The model I'm thinking of is really about someone saying "I'm going to go > work on . Please don't work on without talking to me first, > because it would be a shame to waste all my time, or all your time, by not > coordinating." I consider this to be more of a lock based on intention > than a "hard lock". It's about avoiding duplicate work, trodden on toes, > etc. It might be akin to claiming a task from a public task list. > > The lock you're describing appears to be more of a source tree lock: no > one touch this section of the source tree, because someone is doing work > relating to it in another tree. For example, someone might ask that no > one touch the USB code for a few days while a change is phased in. Or > that people expect the tree to break for a few days as a large API change > is phased in and the final integration tested. This is not the kind of > lock I'm talking about. But you see I am doing KSE despite the fact that other people are changing and improving code that is central to KSE all the time. When it happens I integrate the change, modify it to handle threads and move on. Even John does this but it's OTHER PEOPLE who keep leaping up and demanding that things not be committed becasue it may give john a headache.. well, his "lock" is to long (and anyhow it's not a lock. It's just what he's working on. it can't be allowed to screw over everybody. > > Robert N M Watson FreeBSD Core Team, TrustedBSD Project > [EMAIL PROTECTED] NAI Labs, Safeport Network Services > > > To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
On Tue, 26 Feb 2002, Garance A Drosihn wrote: > That would be me... > > I meant "lock" in the sense of expecting no one to make any major > changes in the same area of code. I seem to remember you asking for > such a "lock" (to use the term loosely) in July, and the KSE work going > in around August or so. My memory is admittedly vague. My point was > you explicitly asked for permission to go ahead with that work, even > though (at the time) we were shooting for a release date in November. > > I don't mean the period of time that -current was actually unstable, but > the amount of time that other developers were asked to stay away from a > major section of code. So it seems to me we have at least three types of locks now, actually: (1) Locks protecting specific bits of code in the tree; for example, during a merge effort that requires several commits over a few days to a week. This should be very short. (2) Locks that protect a subsystem from substantial changes, such as a request to not change vm_mmap.c for a week or two during a rewrite: small changes might be fine, but larger ones would substantially hamper the work. These should also be short. (3) Locks that protect a particular task on a particular piece of code. These simply assert "Don't do this particular activity, becasue I'm doing it". My feeling is that these should be permitted to be long, and are in fact desirable. They wouldn't protect against changes in the subsystem, just identical changes in the subsystem, subject to common sense. Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
>>Some things are too impractically large to do incrementally and are an >>all-or-nothing thing. I recall seeing your early VM commits which were huge, >>you had been working on for months, and were not incremental things. > > Actually, most VM system work that was done was developed over a period of >a few weeks at most, and in most cases were developed and tested in less than >a week. John Dyson had some stuff that brewed for a month or so and in fact >that caused some problems when I wanted to work in a similar area. In those >cases, John D and I collaborated very closely on the VM work, and often >emailed each other patches to integrate into each other's trees. ...but the >important point is that I don't believe that we ever told people not to work >on the VM system because it might conflict with our work. We told people not >to work on it because it was too delicate and too easily broken. :-) Oh, and one more thing... It was ALWAYS forefront in our minds to get any work that we had done committed as soon as possible, specifically to avoid having to deal with conflicts that other people may make, to avoid bit-rot in our working kernel trees, and to get the changes out to the masses for maximal testing. I continue to feel strongly that this is the only way to be successful in developing for FreeBSD. Now, I'm talking about changes to existing files in FreeBSD of course. People that bring whole new arch ports to the kernel have a different problem, but in those cases the overlap with existing files in FreeBSD is very minimal, so longer delays to commit would not normally affect anyone else. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
>>Anyway, my point is that the Perforce repo itself isn't the problem. The >> problem is that people are maintaining private patch sets for long periods >> and making claims to the areas that their patches cover. Step-wise evolution >> is the only way to go in this distributed development model and in order to >> acheive this, private development trees need to be minimized as much as >> possible. > >Some things are too impractically large to do incrementally and are an >all-or-nothing thing. I recall seeing your early VM commits which were huge, >you had been working on for months, and were not incremental things. Actually, most VM system work that was done was developed over a period of a few weeks at most, and in most cases were developed and tested in less than a week. John Dyson had some stuff that brewed for a month or so and in fact that caused some problems when I wanted to work in a similar area. In those cases, John D and I collaborated very closely on the VM work, and often emailed each other patches to integrate into each other's trees. ...but the important point is that I don't believe that we ever told people not to work on the VM system because it might conflict with our work. We told people not to work on it because it was too delicate and too easily broken. :-) -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
On Tue, 26 Feb 2002, Julian Elischer wrote: > > On the other hand, you could easily argue that the expectations might be > > much lower for smaller pieces of work. For example, the move to td_ucred > > required a substantial amount of infrastructure, but the patches > > themselves are relatively sane and small. Once the pieces are all in > > place (which they mostly are), knowing that someone has a lock on it is > > probably worth a "are you still working on this" followed by a wait of a > > week or two to see if it turns up before forging ahead. > > Bad choice.. > p->p_ucred => td->td_ucred in istself requires almost no > infrastracture change, the bulk of the commit is trivial, (AND STILL NOT > COMMITTED) and waiting for weeks on such a trivial thing before being able > to proceed on some interlocking piece is foolish. I was referring to its dependency on KSE. > > (1) The timeout begins when contention occurs, of the lock has been > > declared. This means that if you seriously intend to do some work, > > you can say "I'm going to do the work", but you don't risk losing the > > lock until someone comes to you and says "Hey did you ever...". > > Locks shouldn't be unilateral and they shouldn't last more that the > amount of time that it takes to import a change and get it stable. i.e. > maybe a week or so at most. Someone used KSE-II as an example.. They > said I had a 2 month lock (??) I don't know where they got that idea > from.. I had a vague concensus that maybe things may be broken for a DAY > OR TWO. while the commit was happenning. I the end it was about right. So it sounds like we disagree on what a lock is. My interpretation of the word lock was that it refered to ownership of a task. For example, we gave you ownership of the general task of pushing KSE forwards. This means that if someone turns up and says "I have SAs, I did it entirely differently, I'm going to commit it now" we might say "Hold just a second there". We'd probably say that even if they said "I have SAs, I did them entirely the same way, I'm going to commit it now". The model I'm thinking of is really about someone saying "I'm going to go work on . Please don't work on without talking to me first, because it would be a shame to waste all my time, or all your time, by not coordinating." I consider this to be more of a lock based on intention than a "hard lock". It's about avoiding duplicate work, trodden on toes, etc. It might be akin to claiming a task from a public task list. The lock you're describing appears to be more of a source tree lock: no one touch this section of the source tree, because someone is doing work relating to it in another tree. For example, someone might ask that no one touch the USB code for a few days while a change is phased in. Or that people expect the tree to break for a few days as a large API change is phased in and the final integration tested. This is not the kind of lock I'm talking about. Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
David Greenman wrote: > >In the past week, a number of comments have been made both for and against > >additional version control mechanisms being used to supplement the FreeBSD > >Project official CVS server. Proponents of additional mechanisms, such as > >It's my view that work that happens outside of our official CVS repo is > private work no matter what source control system is used (or if one is used > at all). It is always a problem for one developer to say "I've got changes to > , so don't touch that part of the system." This might be justified > for a very short period (measured in a few days at most), but anything longer > term will almost certainly put off other developers that may wish to work > in the same or related area. >This isn't a new problem, either. We've had similar turf conflicts before > that very much resembled the one at hand. So...What's that phrase? - "Either > take a dump or get off the pot". Something like that. Developers need to > develop in ways that their work gets out as soon as possible so that 1) Other > developers can review the work as it happens, make comments - perhaps > influencing the design at an early stage when it really needs to be, and > 2) So that you don't end up with some buggy mega-commit that has so much > stuff wound up in it that it's nearly impossible to isolate the bugs. >Anyway, my point is that the Perforce repo itself isn't the problem. The > problem is that people are maintaining private patch sets for long periods > and making claims to the areas that their patches cover. Step-wise evolution > is the only way to go in this distributed development model and in order to > acheive this, private development trees need to be minimized as much as > possible. Some things are too impractically large to do incrementally and are an all-or-nothing thing. I recall seeing your early VM commits which were huge, you had been working on for months, and were not incremental things. I'm not taking sides on what has been done offline in the current fights with this statement, but I do take issue with statements that everything can and should be done incrementally. Take the sparc64 stuff. For quite some time, they had hacks in the generic PCI code that was deemed unacceptable to the rest of the platforms. The trick to things working smoothly is to not overuse stuff out of the cvs tree. Cheers, -Peter -- Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms(fwd)
On Tue, 26 Feb 2002, Robert Watson wrote: > > On Tue, 26 Feb 2002, Garance A Drosihn wrote: > > > On the other hand, you could easily argue that the expectations might be > much lower for smaller pieces of work. For example, the move to td_ucred > required a substantial amount of infrastructure, but the patches > themselves are relatively sane and small. Once the pieces are all in > place (which they mostly are), knowing that someone has a lock on it is > probably worth a "are you still working on this" followed by a wait of a > week or two to see if it turns up before forging ahead. Bad choice.. p->p_ucred => td->td_ucred in istself requires almost no infrastracture change, the bulk of the commit is trivial, (AND STILL NOT COMMITTED) and waiting for weeks on such a trivial thing before being able to proceed on some interlocking piece is foolish. Further more as td_ucred can be used almost everywhere p_ucred is used, and p_ucred is not going away, it could have been done piecemeal without any danger.. I could change one occurance of p_ucred to be td->ucred pehour over a 2 month period whenever I had a few spare cycles and the tree would be none the worse for it. > > (1) The timeout begins when contention occurs, of the lock has been > declared. This means that if you seriously intend to do some work, > you can say "I'm going to do the work", but you don't risk losing the > lock until someone comes to you and says "Hey did you ever...". Locks shouldn't be unilateral and they shouldn't last more that the amount of time that it takes to import a change and get it stable. i.e. maybe a week or so at most. Someone used KSE-II as an example.. They said I had a 2 month lock (??) I don't know where they got that idea from.. I had a vague concensus that maybe things may be broken for a DAY OR TWO. while the commit was happenning. I the end it was about right. > > (2) The strength (timeout) of the lock depends on the level of investment > by the person holding the lock. That means if it's a trivial patch, > the time to break the lock is short, perhaps nil, but if it's a large > piece of work, there's the opportunity to say "Yes, it's a work in > progress, is that OK?", "Give me a week to finish it up", or "It's > going to take me a long time, why don't you take over", or in the > worst case "I'm doing it" followed by a timeout. > > (3) Common sense applies. > > It also suggests that there be a way to register intent and interest > relating to a topic. For example, I maintain a web page expressing intent > and on-going work regarding the TrustedBSD Project. It's linked from the > FreeBSD projects page. I've posted about it on lists. That suggests that > I have a relatively strong lock, for some definition of lock. It doesn't > preclude me from accepting contributions, soliciting contributions, etc. > It just means that if someone says "Ok, that's it, enough waiting, where > is it" I probably get a little more patience because people are aware the > work is on-going, before I get timed out and blown off. > > Robert N M Watson FreeBSD Core Team, TrustedBSD Project > [EMAIL PROTECTED] NAI Labs, Safeport Network Services > > > > To Unsubscribe: send mail to [EMAIL PROTECTED] > with "unsubscribe freebsd-current" in the body of the message > To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
>In the past week, a number of comments have been made both for and against >additional version control mechanisms being used to supplement the FreeBSD >Project official CVS server. Proponents of additional mechanisms, such as It's my view that work that happens outside of our official CVS repo is private work no matter what source control system is used (or if one is used at all). It is always a problem for one developer to say "I've got changes to , so don't touch that part of the system." This might be justified for a very short period (measured in a few days at most), but anything longer term will almost certainly put off other developers that may wish to work in the same or related area. This isn't a new problem, either. We've had similar turf conflicts before that very much resembled the one at hand. So...What's that phrase? - "Either take a dump or get off the pot". Something like that. Developers need to develop in ways that their work gets out as soon as possible so that 1) Other developers can review the work as it happens, make comments - perhaps influencing the design at an early stage when it really needs to be, and 2) So that you don't end up with some buggy mega-commit that has so much stuff wound up in it that it's nearly impossible to isolate the bugs. Anyway, my point is that the Perforce repo itself isn't the problem. The problem is that people are maintaining private patch sets for long periods and making claims to the areas that their patches cover. Step-wise evolution is the only way to go in this distributed development model and in order to acheive this, private development trees need to be minimized as much as possible. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
On Tue, 26 Feb 2002, Garance A Drosihn wrote: > I think the main issue here is how long the real repository can be > "locked" while waiting for some change to show up. If work can keep > going into the main repository, then what does anyone care if someone is > tracking their own personal work using CVS, or perforce, or a bunch of > home-grown shell scripts? I think the answer varies a great deal by the work that's being done. Suppose it's an implementation of mandatory access control on FreeBSD. Would it be reasonable for me to request that anyone who is interested in doing MAC on FreeBSD talk to me first before starting, so we can sync up and make sure we're coordinating, even if it's going to take 16 months to implement? Probably. Especially given that we'll probably end up investing at least four man years (probably more) developing it, meaning that people are unlikely to want to replicate that work when they could just wait :-). On the other hand, you could easily argue that the expectations might be much lower for smaller pieces of work. For example, the move to td_ucred required a substantial amount of infrastructure, but the patches themselves are relatively sane and small. Once the pieces are all in place (which they mostly are), knowing that someone has a lock on it is probably worth a "are you still working on this" followed by a wait of a week or two to see if it turns up before forging ahead. Imagine it's an initial port to IA64. If someone has announced they're working on it, you might expect people who would like to assist to checkpoint, and even wait a couple of months if substantial work has been done, before cutting it from scratch. Imagine it's an announced set of patches to clean up some #define's in a single C file. That's probably worth a couple of days wait to see if they're serious about doing it, since it's unlikely to stand in the way of much, but probably not much more. This suggests three rules of thumb that might make sense: (1) The timeout begins when contention occurs, of the lock has been declared. This means that if you seriously intend to do some work, you can say "I'm going to do the work", but you don't risk losing the lock until someone comes to you and says "Hey did you ever...". (2) The strength (timeout) of the lock depends on the level of investment by the person holding the lock. That means if it's a trivial patch, the time to break the lock is short, perhaps nil, but if it's a large piece of work, there's the opportunity to say "Yes, it's a work in progress, is that OK?", "Give me a week to finish it up", or "It's going to take me a long time, why don't you take over", or in the worst case "I'm doing it" followed by a timeout. (3) Common sense applies. It also suggests that there be a way to register intent and interest relating to a topic. For example, I maintain a web page expressing intent and on-going work regarding the TrustedBSD Project. It's linked from the FreeBSD projects page. I've posted about it on lists. That suggests that I have a relatively strong lock, for some definition of lock. It doesn't preclude me from accepting contributions, soliciting contributions, etc. It just means that if someone says "Ok, that's it, enough waiting, where is it" I probably get a little more patience because people are aware the work is on-going, before I get timed out and blown off. Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Discussion of guidelines for additional version control mechanisms (fwd)
Apparently a number of people missed this post, so I'm resending. To recap: this is an attempt to brainstorm for ideas about how we can improve our use of version control, while responding to concerns about access the resulting work. The goal is to formulate a set of guidelines based on whatever suggestions come out of this work. Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services -- Forwarded message -- Date: Sat, 23 Feb 2002 16:28:47 -0500 (EST) From: Robert Watson <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Subject: Discussion of guidelines for additional version control mechanisms In the past week, a number of comments have been made both for and against additional version control mechanisms being used to supplement the FreeBSD Project official CVS server. Proponents of additional mechanisms, such as Perforce, have pointed out that CVS doesn't provide the necessary tools to support several important development models, including those based on light-weight branching and three-way merges. Others have pointed out that while these technical benefits might be real, the resulting decentralization can be a problem for the project, and that requiring additional version control mechanisms in order to work with the project is simply not feasible. For the purpose of this discussion, please assume that additional version control facilities serve a useful purpose (by supporting collaboration, tree tracking, etc), but that no everyone will be able to use them, both based on preference, and for technical and material reasons. The purpose of this message is to initiate a serious discussion of what guidelines might be put in place to help facilitate the use of additional version control mechanisms (which are inevitable, even if it means only individual developers with local CVS repositories of their own), and how to specifically address the problems that have been posed, in particular relating to communication. The outcome of this conversation will probably not be a set of rules: there is sufficient variation in the nature of various sub-projects that it wouldn't make sense. However, it will result in a set of recommendations to maximize communication and acceptance by the broader community. I've mixed in some suggested things to think about as possible answers, but there are probably many more things to consider. Question 1: How should the presence of on-going work in an external repository be announced to the broader community? Suggestion: e-mail to -arch, -current, or a related mailing list Question 2: How should the status of on-going work be announced to the broader community? Suggestion: Bi-monthly developer status report Suggestion: Status web page for the project Suggestion: Regular status reports on work to relevant mailing lists Question 3: How should the results of the on-going work be made available to the broader community? Suggestion: cvsup10 export of the Perforce tree Suggestion: patchsets sent to appropriate mailing lists with status Suggestion: patchsets generated automatically and posted to the mailing list Question 4: How agressively should on-going work be pushed back into the base tree? (Note that the answer here depends a great deal on the nature of the work: both its rationale, its goals, etc). In particular note that currently Perforce is used to hold experimental changes, as well as ones destined for eventual production use. Suggestion: For work requiring large source tree sweeps, API changes, etc, only when the work is ready to commit. Suggestion: For more minor work, P4 might be used only for brief collaboration, or to assist in patch preparation. It's worth noting before getting into too much discussion that there's a spectrum of possibilities, and different answers may be appropriate for different circumstances. If P4 is not used as a collaboration tool, only for local version management for individual developers, it serves the same purpose as any local source tree. Likewise, at low levels of collaboration, it's no different from mailing patchsets. As the level of collaboration increases, the failed balancing point seems to be in providing access to non-P4 users. It might be useful to look at this discussion through a series of case examples, including projects such as KSE, SMPng, OpenPAM, TrustedBSD, architecture ports such as ia64, sparc64, etc. Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message