Why not using CVSS as a base calculation for assigning severity levels? IIRC, something like:
CVSS>=8 => High 4<=CVSS<8 => Medium CVSS<4 => Low was a good guidance in my previous job. FYI, I've attached the table that drove us to these score. Cyrille Le mercredi 10 avril 2024 à 23:30 +0200, Ola Lundqvist a écrit : > Hi again > > I have started with a document that clarify the severity levels. I > also introduced the level "critial" but I'm not sure it adds any > value. > > https://inguza.com/document/debian-security-severity-levels > > This is just a first draft. It is not final. But comments are > welcome. > > // Ola > > On Wed, 10 Apr 2024 at 22:03, Ola Lundqvist <o...@inguza.com> wrote: > > > > Hi Raphael > > > > You can see corrected statistics in a separate email. > > > > Now to comment a few things below. > > > > On Wed, 10 Apr 2024 at 10:49, Raphael Hertzog <hert...@debian.org> > > wrote: > > > > > > Hello, > > > > > > On Tue, 09 Apr 2024, Ola Lundqvist wrote: > > > > Let me use some data from CVEs for last year 2023. > > > > I used the following method to extract the data > > > > grep -B 5 '\[buster\]' list | grep -A 5 "^CVE-2023-" | grep > > > > '\[buster\]' > > > > and then grepped for the end-of-life, not-affected (and so on > > > > to count further). > > > > It is not a perfect method, but I think it is good enough to > > > > crunch the numbers. > > > > > > > > 1260 CVEs with tag [buster] > > > > 382 end-of-life > > > > 410 not-affected > > > > 281 no-dsa > > > > 71 postponed > > > > 58 ignored > > > > 58 fixed (most in the linux kernel) > > > > > > Those numbers are quite surprising. I hope there's some error > > > somewhere > > > otherwise I wonder what has been done in the 2400+ hours paid > > > each year to > > > work on LTS... I'm pretty sure we have fixed more than 58 CVE. > > > The average > > > month has 20 to 30 updates (see > > > https://lists.debian.org/debian-lts-announce/2024/03/threads.html > > > for > > > example). > > > > Yes I was surprised as well. When I checked yesterday I could not > > find > > the fault but today with fresh eyes it was rather obvious. :-) > > Sorry for the confusion. See the other email about that. > > > > > > A large portion of the CVEs are marked as no-dsa for a long > > > > time. We > > > > still have 200 of them from year 2019 for example. > > > > > > > > And I think this is a good practice. Security is always a > > > > balance > > > > between a lot of things. Fixing everything will likely result > > > > in a lot > > > > of tools breaking. > > > > Also following the regular Debian Security team is also a good > > > > practice. They do a good job of analyzing things and we should > > > > normally not fix issues in buster when it will not be fixed in > > > > later > > > > releases. So this makes a lot of sense. > > > > > > This comment still conflates "no-dsa" with "doesn't need to be > > > fixed". We > > > clearly told you many times that it doesn't mean that at all. > > > > Did I write that? I cannot see it it what you comment. But after > > reading the whole email I think I understand why you think so. > > If you read my proposal I suggest we postpone all no-dsa and then > > wait > > for point release updates as one way to handle it. > > That is basically the same thing as what you write below, that no- > > dsa > > means no need for an urgent fix. > > > > But then you commented that we should do our own judgement. Yes it > > would be good if we have a guilde. More about that below. > > > > > "no-dsa" means "doesn't need to be urgently fixed, and will not > > > be fixed > > > by the security team". It basically defers the question to the > > > package > > > maintainer. It's up to the package maintainer to decides between > > > "to fix via spu", "postponed" or "ignored". > > > > I agree to this. > > > > > Some package maintainers will typically decide to fix it via a > > > point > > > release. But they rarely update the triaging to document > > > "postponed" or > > > "ignored". So that's why it's up to the LTS team to make that > > > call > > > when we are (alone) in charge of a given release. > > > > This is a good point. I had missed that package maintainers do not > > update the security tracker. Can we have tools to help us with > > this? > > > > > And regarding "we should normally not fix issues in buster when > > > it will > > > not be fixed in later releases", we have already explained that > > > it goes > > > the other way around: if we decide that something is worth to be > > > fixed > > > in buster, then we should help to get it fixed in later releases. > > > > I do agree with that approach. The question then is when we should > > do that. > > I think we need some type of guidance. It is very easy to fall into > > extremes. > > One extreme is to only follow secteam. > > The other extreme is to try to fix all no-dsa no matter how > > unimportant it is. > > > > > Obviously if the sec-team triages it as unimportant/ignored, then > > > we are > > > unlikely to fix it. But when the sec team opts for no-dsa, then > > > we are > > > entitled to decide to fix it and we can also prepare the SPU to > > > fix newer > > > releases. > > > > Yes, then the question is what rule/guide to follow. We can't fix > > all. > > That is clear from the statistics. > > > > > > 2) Another way is to use the definition we have of > > > > unimportant/low/medium/high classification and use that as > > > > judgement. > > > > > > I wanted to comment about this classification. While it's not in > > > active > > > use by the security team, I think it would be nice if we could > > > provide > > > such a classification: > > > > Yes, I think so. Or some other method we choose to follow. > > But I think we need to clarify it, because it is a little fuźzy at > > the moment. > > > > > 1/ it can help to drive attention to the the most pressing CVE > > > 2/ it lets us monitor how well we are performing for different > > > severities > > > (in terms of delay to provide a fix) > > > > > > Clearly the LTS sponsors who are funding security work would like > > > to see > > > some things like "90% of the high-priority CVE are fixed within > > > one week". > > > But we are not able to provide any data. > > > > Precisely. > > > > > Also we are regularly telling customers/sponsors that we are > > > doing on own > > > assessment of the importance of CVE and that it may diverge from > > > the > > > standard CVSS score that they use as reference. In the ideal > > > world, we > > > would provide our own CVSS score with explanations but we are > > > quite far > > > from this. In the mean time, it would already be great if we had > > > more > > > reliable low/medium/high scoring... > > > > Yes it would be really good. If we add this as part of the triaging > > work it would be good. > > In a way we need to do that already because we say that we should > > not > > simply tag no-dsa, but rather postpone or not. > > > > There are a few things we need to decide on: > > - What severity levels warrants the package to be in dla-needed? Is > > low one of them? > > - Should all medium be in dla-needed? > > > > If you ask me I think the line is somewhere in medium, but I'm not > > sure of what definition to use yet. > > > > Cheers > > > > // Ola > > > > > > > > > > Cheers, > > > -- > > > ⢀⣴⠾⠻⢶⣦⠀ Raphaël Hertzog <hert...@debian.org> > > > ⣾⠁⢠⠒⠀⣿⡁ > > > ⢿⡄⠘⠷⠚⠋ The Debian Handbook: > > > https://debian-handbook.info/get/ > > > ⠈⠳⣄⠀⠀⠀⠀ Debian Long Term Support: https://deb.li/LTS > > > > > > > > -- > > --- Inguza Technology AB --- MSc in Information Technology ---- > > > o...@inguza.com o...@debian.org | > > > http://inguza.com/ Mobile: +46 (0)70-332 1551 | > > --------------------------------------------------------------- > > >
score.pdf
Description: Adobe PDF document
