On 10/09/2025 16:28, Paul Lodge wrote:
Hi All,
if you decide to have CVE tests made public I would suggest 1 small
caveat, that the fixes should older than x number of months,
that would permit users to upgrade their systems.
This was one of the things we discussed in Bratislava.
There were lots of views. From memory some of those were:
- It depends on how complex it is to derive the attack from the CVE. The
simpler it is, the sooner we should/could add the unit test.
- Once the CVE is public, it is better to have the test out ASAP. It
helps users determine if they are vulnerable more than it helps
attackers. It also helps downstream distributions test their
backports.
- Don't release the test at all. Anything we can do to make an
attacker's job harder is a good thing.
And lots of other positions around / between those.
It is tricky. I could argue for any of those positions. Overall, I think
the answer is a combination of:
- it is better for these tests to be public (at some point) than not
- exactly when to make a test public is a judgement call that needs to
be made on a case by case basis
- encouraging users to keep up to date with the latest release is a good
thing
Mark
Best
Paul
On 10/09/2025 14:18, Coty Sutherland wrote:
On Wed, Sep 10, 2025 at 7:23 AM Dimitris Soumis <[email protected]>
wrote:
On Wed, Sep 10, 2025 at 12:15 PM Mark Thomas <[email protected]> wrote:
All,
One of the topics at the security day we held in Bratislava was adding
unit tests for CVEs once the CVEs were public.
I have just rediscovered a test case for CVE-2025-53506 sat in a git
stash it would be good to get committed.
Before I commit anything, I was wondering how we wanted to organise
these. Options include:
- just another test in the relevant class
- dedicated CVE test classes alongside the standard test classes
- a dedicated package for CVE tests
I was thinking a new, dedicated package:
org.apache.tomcat.security
One class per year e.g.:
TestSecurity2025
TestSecurity2024
...
+1 for the dedicated package. It will be good to have CVE related tests
organised, as it will be easier to discover, maintain and enhance.
Though a possible concern of concentrating CVE tests would be that we
lower
the bar for one to discover edge cases or gaps in fixes.
One (or more tests per CVE)
public void testCVE_2025_53506()
or
public void testCVE_2025_53506a()
public void testCVE_2025_53506b()
or
public void testCVE_2025_53506_01()
public void testCVE_2025_53506_02()
...
+1 testCVE_YYYY_NNNNN[_nn]()
I'm not expecting every CVE to get a test case but, where we have them,
I think it makes sense to make them known and available. This is also
something we can add to over time. I suspect there are a few existing
tests that are for known CVEs but were never marked as such.
+1 in moving pre existing CVE focused tests to the dedicated package.
Thoughts?
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
I am already working on creating tests for CVEs that do not have a test
scenario in the commit that points to the fix.
Some extra thoughts:
1) We add the link to the fix commit of each CVE as well as to the CVE
itself in the test classes, as it is in
https://tomcat.apache.org/
security-10.html#Fixed_in_Apache_Tomcat_10.1.44,
for
traceability.
2) Avoid over explanatory comments in code that will make easier for
someone to discover scenarios we haven't considered.
+1 from me to all of Dimitris' comments
Kind regards,
Dimitris
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
