[Reportbug-maint] Bug#878088: Bug#878088: reportbug: please inform security and lts teams about security update regressions

[Nis Martensen]

On 26-01-2018 15:45, Markus Koschany wrote:
> I am not convinced that the apt-cache method is more efficient than
> parsing the version string. I believe my method is simpler and it would
> catch the same potential candidates as your apt-cache idea. Manual
> intervention (answering a question) cannot be avoided unless the
> security team agrees to receive all bug reports against a version with a
> security update. I am absolutely sure that is not desired.

I agree that the question should be asked when the package version is a
security update.  What I am trying to achieve using the apt-cache method
(on top of the version string method) is to avoid asking the question
for "normal" package updates in stable.

Attached a new version of the is_security_update function. This could be
further refined by fetching the changelog from the package tracker if
the package version is not the installed one, but this is probably going
too far...

No idea how many of the stable package updates are usually normal bug
fix updates compared to the number of security updates. If updates are
almost all security updates, then we should definitely not do such
micro-optimization and go with your original approach.


> I favor my current patch because of the reasons I mentioned before. I
> can remove the sys.exit call? What else should be done?

"secversion[2]" should be "secversion.group(2)", right?  The former
variant did not work for me in a quick test.

Using an else clause may be more pythonic than my previous suggestion of
moving more stuff into the try block.

Reportbug has an "ewrite()" function that you can use for the warning
message.

Reportbug has a concept of user expertise levels.  Can the question be
skipped in novice mode?

Should reportbug incorporate a default version of the json file to fall
back to if the lookup fails? Reportbug is probably going to be updated
more often than the online version of the json file. An internal version
could also be updated regularly.
>From 56d6dbffd3075bb9f06aa1b31fd8f688b6b16d4b Mon Sep 17 00:00:00 2001
From: Nis Martensen 
Date: Sat, 27 Jan 2018 22:22:31 +0100
Subject: [PATCH] utils: add new is_security_update() function

The security and LTS teams want to ask the reporting user for
notification if one of their updates caused a regression.  On the other
hand, reportbug should not ask users unnecessary questions.  So let's
try to collect evidence that this report is really about a package
version from a security update.
---
 reportbug/utils.py | 79 ++
 1 file changed, 79 insertions(+)

diff --git a/reportbug/utils.py b/reportbug/utils.py
index b376c6c..c372727 100644
--- a/reportbug/utils.py
+++ b/reportbug/utils.py
@@ -39,6 +39,8 @@ import email
 import socket
 import subprocess
 import pipes
+import apt
+import gzip
 
 from .urlutils import open_url
 from string import ascii_letters, digits
@@ -1333,3 +1335,80 @@ def get_lsm_info():
 break
 
 return lsminfo
+
+
+def is_security_update(pkgname, pkgversion):
+"""Determine whether a given package is a security update.
+
+Detection of security update versions works most reliably if the
+package version under investigation is the currently installed
+version.  If this is not the case, the probability of false
+negatives increases.
+
+Parameters
+--
+pkgname : str
+package name
+pkgversion : str
+package version
+
+Returns
+---
+bool
+True if there is evidence that this version is a security
+update, otherwise False
+"""
+# Check 1:
+# If it does not follow the debXuY version number pattern, it is
+# definitely no security update.
+#
+# This check is not sufficient to detect security updates reliably,
+# since other stable updates also use the same version pattern.
+regex = re.compile('(\+|~)deb(\d+)u(\d+)')
+secversion = regex.search(pkgversion)
+if not secversion:
+return False
+
+# Check 2:
+# If the package comes from the Debian-Security package source, it
+# is definitely a security update.
+#
+# This check does not identify all security updates, since some of
+# them are distributed through the normal channels as part of a
+# stable release update.
+try:
+p = apt.Cache()[pkgname]
+if 'Debian-Security' in [o.label for o in
+p.versions[pkgversion].origins]:
+return True
+except:
+pass
+
+# Check 3:
+# Inspect the package changelog if it mentions any vulnerability,
+# identified by a CVE number, in the section of the latest version.
+
+cl = None
+for cl in ['/usr/share/doc/{}/changelog.Debian.gz'.format(pkgname),
+   '/usr/share/doc/{}/changelog.gz'.format(pkgname)]:
+if os.path.exists(cl):
+break
+
+try:
+with gzip.open(cl, 'rt') as f:
+

[Reportbug-maint] Bug#878088: Bug#878088: reportbug: please inform security and lts teams about security update regressions

[Markus Koschany]

Am 26.01.2018 um 05:43 schrieb Sandro Tosi:
[...}
> i like the idea of trying hard to avoid to ask questions to the users
> so maybe we can do something like
> 
> * check if that version is coming from the debian-security repo
> ** if so, copy the relevant security team
> ** if not, ask the user

I try to respond to all the ideas that were mentioned by Nis and you to
avoid the question to end users.

We have two different teams. The LTS team and the security team. The
security team is responsible for stable and oldstable security updates
while the LTS team takes care of oldoldstable. I see several issues with
the apt-cache approach.

If we check whether the update is coming from the debian-security repo,
then only the security team is concerned. All LTS updates are from
oldoldstable or from the distribution codename (at the moment this is
Wheezy).

If I implemented your suggestion then all bug reports for a
"Debian-Security" update would be automatically copied to the security
team's mailing list. As I have pointed out before, it is possible that
the user just wants to report a "normal" bug in a version with a
security update which is completely unrelated to a security update. The
same reasoning applies to the idea of checking for "oldoldstable". The
only way to avoid unwarranted copies is to ask the user directly. There
might still be false-positives but there should be far less noise in
comparison to

if Debian-Security:
  CC security team
else ask user


Replying to Nis' questions:

> This requires more effort.  Does the package tracker offer a way to
> query such information?  The only other idea I have right now involves
> inspecting the latest entry in changelog.Debian.gz. ("Was the package
> uploaded by the maintainer or one of the normal uploaders?")  Do you
> have other ideas on how a user might know whether a package update
> delivered in a stable point release was a security update?

We have the UDD database that provides a plethora of information about
packages in all distributions.

https://wiki.debian.org/UltimateDebianDatabase/

A PostgreSQL database query is required though. We have already
discussed in this bug report that a https request is simpler and better.
We could also parse the changelog for strings like CVE-2018-. A
security update should always include the CVE identifier.

Again these would be alternative methods to determine whether a package
has received a security update. It does not solve the problem whether
the user wants to report a normal bug instead.

> Would it be feasible to make all security updates available via the
> security update channel?

No. Low priority updates are released via a stable/oldstable point
update. LTS updates are also handled differently via oldoldstable.

>  Then the simple suggested method would be
> sufficient.  But it is probably infeasible, otherwise it would be done?

If there was one channel for all security updates you would be correct.
But even then I don't see an advantage compared to parsing the version
string like I do. It is simpler and catches all relevant updates already.

> If there is no good way, maybe asking your question only for the
> packages identified by the proposed method would be acceptable as a
> first step, until a reliable approach is developed?

I am not convinced that the apt-cache method is more efficient than
parsing the version string. I believe my method is simpler and it would
catch the same potential candidates as your apt-cache idea. Manual
intervention (answering a question) cannot be avoided unless the
security team agrees to receive all bug reports against a version with a
security update. I am absolutely sure that is not desired.

> But perhaps Sandro may even be willing to accept a patch based on your
> original version string pattern matching, if his other concerns are
> addressed.  Sandro, what do you think?

I favor my current patch because of the reasons I mentioned before. I
can remove the sys.exit call? What else should be done?

> in neither case is acceptable to sys.exit() if you cant connect to the
> internet: either you decide a default address for this case, or print
> a warning message that you cant fetch the needed information and the
> sec team wont be copied in the repo.

> thanks both for working together on reaching consensus

I can move the code in the try-block as suggested by Nis and simply drop
the sys.exit call. I am already printing a warning message. Is using
print sufficient in this case or is there a better method to visualize
the error? I can extend the message a bit and mention our mailing list
address.

Markus



signature.asc
Description: OpenPGP digital signature
___
Reportbug-maint mailing list
Reportbug-maint@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reportbug-maint