Release team resources

2008-09-22 Thread Jo Rhett
Thank you for answering the resources problem in detail, I appreciate  
it.



For what secteam support of the base system costs, it's mainly time
for the members of the security team which is the cost.  The more
branches, the more time is required.  This is not a linear cost and
has multiple parts to it:

...

There is also a cost in hardware for supported branches though this is
less of an issue.

...


The more releases are supported, the more disk-space is also needed
for freebsd-update mirrors.  Again, far from an unsolvable problem by
any means, but also a factor


This is what I suspected, but having the detail backing it up helps  
tremendously.


Has there been done any work on metrics for the support needs?   
Obviously these are a bit of hand waving because the number and type  
of security problems are hard to predict, but it does help to provide  
a useful model for understanding costs


In specific, is it known how many man-hours would be necessary to  
extend support for a recent release?


NOTE that I am not trying to extend the support for 4.x or 5.x or even  
6.x once 8 has shipped.  I think that 2 full releases is perfectly  
reasonable.  I'm just asking about the recent releases.



While I'm not going more into the general discussion of how long to
support branches, I will note that as rwatson has said - adding more
people to secteam is not as simple as it sounds (though we are in the
process of expanding right now).


I assumed not.  I was curious to what extent outside people could help  
support the process, while leaving commits to the internal people.   
For example, for everything except the jail vulnerability in the last  
4 years the security problems were related to third party utilities,  
and were widely published in security mailing lists.  Someone without  
a commit bit could certainly build the patch, test the patch on  
relevant versions, etc.


Likewise, if a patch was created for the latest version, an outside  
person could test the patch on a desired-to-support build, confirm  
that it works and/or change the patch as necessary for the older build  
(like when third party utility versions were different between major  
releases).


Is the overhead of supporting these not-committers such that it is  
not useful for the secteam as a whole?


(obviously the longer term goal would be to determine which of the  
outside testers would be useful for promoting within the group)



Newer patches also wouldn't make it to freebsd-update
as that is managed by secteam.


For my needs/desires I'd rather focus on something that would get  
pushed to freebsd-update.



We have had one case where a committer was interested in supporting an
older release and back-ported patches from security advisories for a
while.  The patches for the older releases were then reviewed in each
case by the security team before commit, but that only lasted for a
while and was a couple of years ago AFAIR.  In theory this could
happen again if the Security Officer at the time is OK with it - I
haven't talked with Colin about this in a while, so I can't recall is
position.  There would still need to be committer which is the
interface to secteam and do the commits.  Most issues (though of
course not all) which gets advisories are not public at the time of
the advisory, so a fix to older branches would be likely be delayed
some compared to initial disclosure.


As noted above, very few of the security releases were based on  
information not available to the general public (who read security- 
related mailing lists, anyway)


--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source  
and other randomness

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Release team resources

2008-09-22 Thread Robert Watson


On Mon, 22 Sep 2008, Jo Rhett wrote:

I assumed not.  I was curious to what extent outside people could help 
support the process, while leaving commits to the internal people.  For 
example, for everything except the jail vulnerability in the last 4 years 
the security problems were related to third party utilities, and were widely 
published in security mailing lists.  Someone without a commit bit could 
certainly build the patch, test the patch on relevant versions, etc.


I'm not sure I agree with this analysis.  From a FreeBSD-centric perspective, 
vulnerabilities fall into four classes:


- FreeBSD-generated code
- Third party code blended with out code (arguably ours also)
- contrib code that is in our revision control
- Ports

We dropped ports from our advisory scope because the number of vulnerabilities 
skyrocketted due to ports growing and the number of vulnerabilities discovered 
in them growing.  We do provide a database of known-vulnerable ports and 
versions, but that's not generally the responsibility of the base security 
team, rather a separate ports security team.  I think this is the right 
trade-off -- among our fears is that we over-release advisories, which would 
devalue the usefulness of advisories over time as referring specifically to 
critical issues.


Extracted from the list of advisories on security.FreeBSD.org going back to 
the beginning of last year:


AdvisoryClass
FreeBSD-SA-08:09.icmp6  Blended
FreeBSD-SA-08:08.nmount FreeBSD
FreeBSD-SA-08:07.amd64  FreeBSD
FreeBSD-SA-08:06.bind   Contrib
FreeBSD-SA-08:05.opensshContrib
FreeBSD-SA-08:03.sendfile   FreeBSD
FreeBSD-SA-08:02.libc   Blended
FreeBSD-SA-08:04.ipsec  Blended
FreeBSD-SA-08:01.ptyFreeBSD
FreeBSD-SA-07:10.gtar   Contrib
FreeBSD-SA-07:09.random FreeBSD
FreeBSD-SA-07:08.opensslContrib
FreeBSD-SA-07:07.bind   Contrib
FreeBSD-SA-07:06.tcpdumpContrib
FreeBSD-SA-07:05.libarchive FreeBSD
FreeBSD-SA-07:04.file   Contrib
FreeBSD-SA-07:03.ipv6   Blended
FreeBSD-SA-07:02.bind   Contrib
FreeBSD-SA-07:01.jail   FreeBSD

Counting on my fingers, that's 7 FreeBSD-specific, 4 that lie in code we 
basically maintain, and 8 that are in externally maintained software.  Seems 
like a pretty even split.  In the case of most third party code 
vulnerabilities, I believe we received non-trivial advanced warning of the 
impending vulnerability announcement.


As noted above, very few of the security releases were based on information 
not available to the general public (who read security-related mailing 
lists, anyway)


I'm not sure I agree with this assertion either.  While there are exceptions, 
most vulnerabilities are known to the security team in advance of public 
discussion.  Depends a bit on which security lists you read, of course...


Robert N M Watson
Computer Laboratory
University of Cambridge
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Release team resources

2008-09-22 Thread Jo Rhett

On Sep 22, 2008, at 1:08 PM, Robert Watson wrote:

I'm not sure I agree with this analysis

...
Counting on my fingers, that's 7 FreeBSD-specific, 4 that lie in  
code we basically maintain, and 8 that are in externally maintained  
software.  Seems like a pretty even split.


Acknowledged, sorry I was working from memory and forgot that the  
things we had to patch for already went through a does it affect us  
filter :-(  You're right.


--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source  
and other randomness



___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]