Solving the great resource problem, take 42 (Re: Upcoming release schedule - 8.4 ?)
On Wed, Jun 13, 2012 at 10:25 PM, Royce Williams royce.willi...@gmail.com wrote: On Wed, Jun 13, 2012 at 8:30 PM, Adrian Chadd adr...@freebsd.org wrote: On 13 June 2012 21:26, Mark Linimon lini...@lonesome.com wrote: On Wed, Jun 13, 2012 at 08:50:24AM -0700, Garrett Cooper wrote: The only way that this would really work is if there were dedicated sustaining engineers working on actively backporting code, testing it, committing it, etc. I'm going to agree with Garrett here. IMHO we've reached (or surpassed) the limit of what is reasonable to ask volunteers to commit their spare time to. This is doubly true when we have more than one stable branch. I totally concur. Ah, but you can get the same effect by freeing up those engineers to work on the hard stuff. This is my usual soapbox (see [1], [2]): Push more of the mundane work out to the edges, so that the developers can focus more on the core (like more releases/features/testing/projects). Here are some ideas. Only developers can implement them, but they would start paying for themselves immediately ... in developer time. - Frequent snapshots, with tools to automatically apply them and roll them back (freebsd-update + ZFS snapshots?). - Tools to do binary walks of snapshots to pinpoint when a bug appeared. (Think 'git bisect' + freebsd-update.) - A taggable FAQ that supports faceted search, and a quick way to add entries (or propose them for approval). - A way to search for known fixes to transient bugs and hardware issues [1]. - General debugging and testing tools for non-developers, including tools for filing smarter bug reports. - A way to automatically upload crash dumps for bulk analysis (like Windows does). - A dmesg analyzer that downloads a list during install, and looks for known issues (or workarounds) with your hardware for that version of FreeBSD (or recommend a different version!). Tools like these would also help more people achieve the I tried it, and it Just Worked moment. This can keep people's interest long enough to give FreeBSD a serious try. Some of them might enter the volunteer pool. I'm not a developer, but if some of the above could be tackled, they might free up enough Developer Equivalents (DEs, a term which I have just made up) to be more than worth the effort. No offense, but speaking from experience, these are referred to as wishlist projects -- many of which get shelved until developers get enough time to work on them. This makes more sense when there are more resources so engineers can work in a less distracted manner as BSD is not Linux as far as BSD's design stratagem is concerned . This is really starting to get philosophical and away from the original intent behind the original post, but given past discussions and the fact that these topics end up going around in circles/cycling through periodically (I've seen it on ports, current/stable/hackers, etc), here's my perspective after having read these discussions a few times and given what I've seen with the project over the last 5-10 years (granted, I've become a jaded realistic/pessimist in past years, so YMMV): Problem Statement (or the I want to have my cake and eat it too Issue): - Users want stability, but want latest and greatest driver code and features. These are [generally] mutually exclusive. Impedance: - There aren't enough volunteer resources to do what consumers of FreeBSD want beyond what's being done, with exception of developers and other volunteers going above and beyond in extraordinary circumstances to get the job done, or doing something his/her day job requires. The former case tends to be more of the exception than the norm. The latter happens sporadically. - There are plenty of companies out there wanting to solve this problem of release cycles, but no one group (or groups) is standing up and actively ponying up dedicated resources on a regular basis to make this a reality. What happens: Trivial tasks, like MFCing, testing, triaging, etc are being done by lead developers in a particular domain, which steals cycles from enhancement/bugfixing work in those areas [or other surrounding areas]; instead of investing time writing regression tests so others can do the work, no one other than the lead developers in a given area can do the work if it requires domain knowledge and/or specific hardware resources to complete the task. Eventually something happens, the developer becomes less active in the community (gets a family, no longer does FreeBSD work, gets a job, number of different things), and depending on the bus factor the particular area being maintained may remain unmaintained for some time. Alternatively, contact is done infrequently enough that interested parties willing to contribute code get discouraged and go off and do their own thing (be it support their own custom distro, switch to another OS, etc). In the former case, there's duplication of effort, some (or most) of which is
Re: Solving the great resource problem, take 42 (Re: Upcoming release schedule - 8.4 ?)
Resending to list, forgot to hit reply-all. On Wed, Jun 13, 2012 at 10:06 PM, Garrett Cooper yaneg...@gmail.com wrote: On Wed, Jun 13, 2012 at 10:25 PM, Royce Williams royce.willi...@gmail.com wrote: On Wed, Jun 13, 2012 at 8:30 PM, Adrian Chadd adr...@freebsd.org wrote: On 13 June 2012 21:26, Mark Linimon lini...@lonesome.com wrote: On Wed, Jun 13, 2012 at 08:50:24AM -0700, Garrett Cooper wrote: The only way that this would really work is if there were dedicated sustaining engineers working on actively backporting code, testing it, committing it, etc. [snip] Ah, but you can get the same effect by freeing up those engineers to work on the hard stuff. This is my usual soapbox (see [1], [2]): Push more of the mundane work out to the edges, so that the developers can focus more on the core (like more releases/features/testing/projects). [my wishlist snipped] No offense, but speaking from experience, these are referred to as wishlist projects -- many of which get shelved until developers get enough time to work on them. This makes more sense when there are more resources so engineers can work in a less distracted manner as BSD is not Linux as far as BSD's design stratagem is concerned . Catch-22. This honestly reads as we can't stop for gas, we're already late. :-) I should have been more clear that I understand that this would require someone to step away from the firehose of work that not having such tools perpetuates. I certainly understand that it requires an effort of will to raise one's head high enough out of the lists/PRs/email swamp long enough, shake off some learned helplessness, and be inspired to tackle one of them. I struggle with that daily myself. There's a Not Invented Here comic strip that is quite applicable: http://notinventedhe.re/on/2010-3-8 [good Garrett summary of the resource problem snipped] So, rather than do things this way by posting wishlist projects that won't happen in the immediate future, why not make developers' lives easier by spreading the load, increasing the domain knowledge in one or more areas, and improving the community in the meantime? Affected companies/the Foundation should have more than enough funds to devote towards a handful of staff to make this a reality, even if the position is part-time. Remember: low hanging fruit - more likely to succeed - quicker/better RoI results. Even one item from my wish list would lower the branches so that more people could reach the fruit. :-) The objections you're raising to my wish list could have been used, in the past, to justify anything from not writing send-pr (which was somewhat low-hanging fruit) to not writing freebsd-update (decidedly less trivial). Not all of my wishlist items require Herculean effort to make progress on. They just require someone who can both code, and see the light at the end of the tunnel that such a project would reveal. It's the never-ending chronic pain and whack-a-mole game of troubleshooting that makes us frame these things as wishes. If we take as an assumption that they're within reach, they might be. It calls to mind the last lines of Say Anything (if I may indulge my John Cusack fanboyhood): Diane Court: Nobody thinks it will work, do they? Lloyd Dobler: No. You just described every great success story. Royce ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Solving the great resource problem, take 42 (Re: Upcoming release schedule - 8.4 ?)
On Wed, Jun 13, 2012 at 11:06:15PM -0700, Garrett Cooper wrote: On Wed, Jun 13, 2012 at 10:25 PM, Royce Williams royce.willi...@gmail.com wrote: On Wed, Jun 13, 2012 at 8:30 PM, Adrian Chadd adr...@freebsd.org wrote: On 13 June 2012 21:26, Mark Linimon lini...@lonesome.com wrote: On Wed, Jun 13, 2012 at 08:50:24AM -0700, Garrett Cooper wrote: The only way that this would really work is if there were dedicated sustaining engineers working on actively backporting code, testing it, committing it, etc. I'm going to agree with Garrett here. IMHO we've reached (or surpassed) the limit of what is reasonable to ask volunteers to commit their spare time to. This is doubly true when we have more than one stable branch. I totally concur. Ah, but you can get the same effect by freeing up those engineers to work on the hard stuff. This is my usual soapbox (see [1], [2]): Push more of the mundane work out to the edges, so that the developers can focus more on the core (like more releases/features/testing/projects). Here are some ideas. Only developers can implement them, but they would start paying for themselves immediately ... in developer time. - Frequent snapshots, with tools to automatically apply them and roll them back (freebsd-update + ZFS snapshots?). - Tools to do binary walks of snapshots to pinpoint when a bug appeared. (Think 'git bisect' + freebsd-update.) - A taggable FAQ that supports faceted search, and a quick way to add entries (or propose them for approval). - A way to search for known fixes to transient bugs and hardware issues [1]. - General debugging and testing tools for non-developers, including tools for filing smarter bug reports. - A way to automatically upload crash dumps for bulk analysis (like Windows does). - A dmesg analyzer that downloads a list during install, and looks for known issues (or workarounds) with your hardware for that version of FreeBSD (or recommend a different version!). Tools like these would also help more people achieve the I tried it, and it Just Worked moment. This can keep people's interest long enough to give FreeBSD a serious try. Some of them might enter the volunteer pool. I'm not a developer, but if some of the above could be tackled, they might free up enough Developer Equivalents (DEs, a term which I have just made up) to be more than worth the effort. No offense, but speaking from experience, these are referred to as wishlist projects -- many of which get shelved until developers get enough time to work on them. This makes more sense when there are more resources so engineers can work in a less distracted manner as BSD is not Linux as far as BSD's design stratagem is concerned . This is really starting to get philosophical and away from the original intent behind the original post, but given past discussions and the fact that these topics end up going around in circles/cycling through periodically (I've seen it on ports, current/stable/hackers, etc), here's my perspective after having read these discussions a few times and given what I've seen with the project over the last 5-10 years (granted, I've become a jaded realistic/pessimist in past years, so YMMV): Problem Statement (or the I want to have my cake and eat it too Issue): - Users want stability, but want latest and greatest driver code and features. These are [generally] mutually exclusive. Impedance: - There aren't enough volunteer resources to do what consumers of FreeBSD want beyond what's being done, with exception of developers and other volunteers going above and beyond in extraordinary circumstances to get the job done, or doing something his/her day job requires. The former case tends to be more of the exception than the norm. The latter happens sporadically. - There are plenty of companies out there wanting to solve this problem of release cycles, but no one group (or groups) is standing up and actively ponying up dedicated resources on a regular basis to make this a reality. What happens: Trivial tasks, like MFCing, testing, triaging, etc are being done by lead developers in a particular domain, which steals cycles from enhancement/bugfixing work in those areas [or other surrounding areas]; instead of investing time writing regression tests so others can do the work, no one other than the lead developers in a given area can do the work if it requires domain knowledge and/or specific hardware resources to complete the task. Eventually something happens, the developer becomes less active in the community (gets a family, no longer does FreeBSD work, gets a job, number of different things), and depending on the bus factor the particular area being maintained may remain unmaintained for some time. Alternatively, contact is done infrequently enough that interested parties willing to contribute code get discouraged and go off
Re: Solving the great resource problem, take 42 (Re: Upcoming release schedule - 8.4 ?)
On Wed, 13 Jun 2012 23:06:15 -0700 Garrett Cooper yaneg...@gmail.com wrote: On Wed, Jun 13, 2012 at 10:25 PM, Royce Williams royce.willi...@gmail.com wrote: On Wed, Jun 13, 2012 at 8:30 PM, Adrian Chadd adr...@freebsd.org wrote: On 13 June 2012 21:26, Mark Linimon lini...@lonesome.com wrote: On Wed, Jun 13, 2012 at 08:50:24AM -0700, Garrett Cooper wrote: The only way that this would really work is if there were dedicated sustaining engineers working on actively backporting code, testing it, committing it, etc. I'm going to agree with Garrett here. IMHO we've reached (or surpassed) the limit of what is reasonable to ask volunteers to commit their spare time to. This is doubly true when we have more than one stable branch. I totally concur. Ah, but you can get the same effect by freeing up those engineers to work on the hard stuff. This is my usual soapbox (see [1], [2]): Push more of the mundane work out to the edges, so that the developers can focus more on the core (like more releases/features/testing/projects). Here are some ideas. Only developers can implement them, but they would start paying for themselves immediately ... in developer time. - Frequent snapshots, with tools to automatically apply them and roll them back (freebsd-update + ZFS snapshots?). - Tools to do binary walks of snapshots to pinpoint when a bug appeared. (Think 'git bisect' + freebsd-update.) - A taggable FAQ that supports faceted search, and a quick way to add entries (or propose them for approval). - A way to search for known fixes to transient bugs and hardware issues [1]. - General debugging and testing tools for non-developers, including tools for filing smarter bug reports. - A way to automatically upload crash dumps for bulk analysis (like Windows does). There's a GSoC project underway for this. - A dmesg analyzer that downloads a list during install, and looks for known issues (or workarounds) with your hardware for that version of FreeBSD (or recommend a different version!). Tools like these would also help more people achieve the I tried it, and it Just Worked moment. This can keep people's interest long enough to give FreeBSD a serious try. Some of them might enter the volunteer pool. I'm not a developer, but if some of the above could be tackled, they might free up enough Developer Equivalents (DEs, a term which I have just made up) to be more than worth the effort. No offense, but speaking from experience, these are referred to as wishlist projects -- many of which get shelved until developers get enough time to work on them. This makes more sense when there are more resources so engineers can work in a less distracted manner as BSD is not Linux as far as BSD's design stratagem is concerned . We have the ideas list for this (http://wiki.freebsd.org/IdeasPage). While it does not attract that much people during the year, it attracts a lot of students which want to participate in the GSoC. Bye, Alexander. -- http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Solving the great resource problem, take 42 (Re: Upcoming release schedule - 8.4 ?)
On Wed, 13 Jun 2012 22:32:08 -0800 Royce Williams royce.willi...@gmail.com wrote: Even one item from my wish list would lower the branches so that more people could reach the fruit. :-) Well... maybe this year for the crashdump auto-submit part. For the rest I suggest to provide some text suitable for the ideas list (see my other reply). Bye, Alexander. -- http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Solving the great resource problem
Spending resources to create more releases is pointless when the PRs aren't getting fixed. Oh, Look! Release 9.2.2.2.2.2 is out! The system still crashes every 5 seconds, but a typo on the true(1) man page is fixed. We need a more global discussion about all the things that resources are spent on, and which are the most useful. Replacing perfectly good components simply because they are GPL. The purpose of BSD is supposed to be creating a great OS, not providing software hoarders with a supply of free code to abuse. Sending people to conferences. Nice, but clearly a luxury. Meanwhile the hardware support is a disaster. PRs sit for years and years and years. The documentation has plenty of room for improvement. It seems there are never enough resources to fix problems, but somehow there are always resources to do yet another fork. FreeBSD, NetBSD, OpenBSD, DragonflyBSD, and now Bitrig. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Solving the great resource problem
On Thu, 14 Jun 2012 15:23:11 -0500, Dieter BSD dieter...@engineer.com wrote: Replacing perfectly good components simply because they are GPL. The purpose of BSD is supposed to be creating a great OS, not providing software hoarders with a supply of free code to abuse. You realize that companies like Juniper have a huge investment in FreeBSD and the less GPL code they have to deal with the better. FYI they give back to us with things like code drops and paid developers. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
