Your message dated Sun, 31 Aug 2014 23:05:01 +0200 with message-id <87a96kwete....@mid.deneb.enyo.de> and subject line Re: Bug#759727: patches for including LTS into security-tracker.d.o has caused the Debian Bug report #759727, regarding patches for including LTS into security-tracker.d.o to be marked as done.
This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact ow...@bugs.debian.org immediately.) -- 759727: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=759727 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
--- Begin Message ---package: security-tracker severity: wishlist tags: patch x-debbugs-cc: debian-...@lists.debian.org Hi, attached are my patches making the security-tracker aware of squeeze-lts. I've tested that in a local instance of the tracker and they work nicely. I think they should be submitted as they are, and as Raphael suggested I send them here for review, I did that. Let me know if I shall commit :) A few comments: $ svn diff|diffstat Makefile | 23 ++++++++++++- fine, I think, I slighlty dislike the variables squeeze_LTS_ARCHS and LTS_MIRROR as well as the update-lts* targets, but it does the trick. bin/check-syntax | 6 ++- bin/tracker_service.py | 2 + bin/update | 2 - bin/updatelist | 2 + lib/python/sectracker/parsers.py | 17 +++++++++ stupid codecopy, but hey, the loader for DTSAs was already a copy of the one for DSAs, so I figured adding one more wasnt too painful ;) lib/python/bugs.py | 47 +++++++++++++++++++++++++-- stupid codecopy, similar to the one in parsers.py... ;) lib/python/sectracker_test/test_analyzers.py | 1 lib/python/sectracker_test/test_parsers.py | 5 ++ lib/python/security_db.py | 35 +++++++++++++------- here I use a trick to make the whole code easier: the release is changed from "squeeze-lts" to "squeeze" and subrelease is set to "lts", so that this matches the "security" suites. the other changes are then straightforward. 10 files changed, 121 insertions(+), 19 deletions(-) That's it. cheers, HolgerIndex: Makefile =================================================================== --- Makefile (Revision 28502) +++ Makefile (Arbeitskopie) @@ -7,6 +7,7 @@ MIRROR = http://cdn.debian.net/debian/ squeeze_ARCHS = amd64 armel i386 ia64 mips mipsel powerpc s390 sparc kfreebsd-i386 kfreebsd-amd64 +squeeze_LTS_ARCHS = amd64 i386 wheezy_ARCHS = amd64 armel armhf i386 ia64 mips mipsel powerpc s390 s390x sparc kfreebsd-i386 kfreebsd-amd64 jessie_ARCHS = amd64 armel armhf i386 mips mipsel powerpc s390x kfreebsd-i386 kfreebsd-amd64 sid_ARCHS = amd64 armel armhf hurd-i386 i386 kfreebsd-i386 kfreebsd-amd64 mips mipsel powerpc s390x sparc @@ -27,7 +28,7 @@ test check: check-syntax check-syntax: stamps/CVE-syntax \ - stamps/DSA-syntax stamps/DTSA-syntax + stamps/DSA-syntax stamps/DTSA-syntax stamps/DLA-syntax stamps/CVE-syntax: data/CVE/list bin/check-syntax $(PYTHON_MODULES) $(PYTHON) bin/check-syntax CVE data/CVE/list @@ -41,6 +42,10 @@ $(PYTHON) bin/check-syntax DTSA data/DTSA/list touch $@ +stamps/DLA-syntax: data/DLA/list bin/check-syntax $(PYTHON_MODULES) + $(PYTHON) bin/check-syntax DLA data/DLA/list + touch $@ + .PHONY: serve serve: @bash bin/test-web-server @@ -136,7 +141,7 @@ done ; \ done -update-old-security: +update-old-security: update-lts for archive in $(OLDSTABLE); do \ for section in main contrib non-free ; do \ $(PYTHON) bin/apt-update-file \ @@ -150,6 +155,20 @@ done ; \ done +LTS_MIRROR = http://ftp.de.debian.org/debian/dists +update-lts: update-lts-$(OLDSTABLE) + +update-lts-$(OLDSTABLE): + set -e && archive=$(shell echo $@ | cut -d- -f3) ; \ + for arch in $($(shell echo $@ | cut -d- -f3)_LTS_ARCHS) ; do \ + $(PYTHON) bin/apt-update-file \ + $(LTS_MIRROR)/$${archive}-lts/main/binary-$$arch/Packages \ + data/packages/$${archive}-lts__main_$${arch}_Packages ; \ + done ; \ + $(PYTHON) bin/apt-update-file \ + $(LTS_MIRROR)/$${archive}-lts/main/source/Sources \ + data/packages/$${archive}-lts__main_Sources ; \ + BACKPORTS_MIRROR = http://ftp.de.debian.org/debian-backports/dists update-backports: update-backports-$(STABLE) update-backports-$(OLDSTABLE) Index: lib/python/security_db.py =================================================================== --- lib/python/security_db.py (Revision 28502) +++ lib/python/security_db.py (Arbeitskopie) @@ -1,4 +1,4 @@ -# security_db.py -- simple, CVE-driven Debian security bugs database +# lts_db.py -- simple, CVE-driven Debian security bugs database # Copyright (C) 2005 Florian Weimer <f...@deneb.enyo.de> # # This program is free software; you can redistribute it and/or modify @@ -385,7 +385,7 @@ AND NOT COALESCE((SELECT NOT vulnerable FROM source_packages AS secp, source_package_status AS secst WHERE secp.name = sp.name - AND secp.release = '%s' AND secp.subrelease = 'security' + AND secp.release = '%s' AND ( secp.subrelease = 'security' OR secp.subrelease = 'lts' ) AND secp.archive = sp.archive AND secst.bug_name = st.bug_name AND secst.package = secp.rowid), 0) @@ -555,6 +555,9 @@ if unchanged: continue + if release == 'squeeze-lts': + release = 'squeeze' + subrelease = 'lts' cursor.execute( """DELETE FROM source_packages WHERE release = ? AND subrelease = ? AND archive = ?""", @@ -615,6 +618,9 @@ raise ValueError, "invalid file name: " + `filename` (release, subrelease, archive, architecture) = match.groups() + if release == 'squeeze-lts': + release = 'squeeze' + subrelease = 'lts' (unch, parsed) = self._parseFile(cursor, filename) unchanged = unchanged and unch for name in parsed.keys(): @@ -726,6 +732,7 @@ sources = ((bugs.CVEFile, '/CVE/list'), (bugs.DSAFile, '/DSA/list'), (bugs.DTSAFile, '/DTSA/list'), + (bugs.DLAFile, '/DLA/list'), (None, source_removed_packages)) unchanged = True @@ -773,12 +780,12 @@ if self.verbose: print " copy notes" - # Copy notes from DSA/DTSA to CVE. + # Copy notes from DSA/DTSA/DLA to CVE. old_source = '' for source, target in list(cursor.execute( """SELECT source, target FROM bugs_xref - WHERE (source LIKE 'DTSA-%' OR source LIKE 'DSA-%') + WHERE (source LIKE 'DTSA-%' OR source LIKE 'DSA-%' OR source LIKE 'DLA-%') AND target LIKE 'CVE-%'""")): if source <> old_source: source_bug = bugs.BugFromDB(cursor, source) @@ -1139,14 +1146,14 @@ # note/release/subrelease triple, but we should check that # here. - status = {'' : {}, 'security' : {}} + status = {'' : {}, 'security' : {}, 'lts' : {}} for (package, note, subrelease, vulnerable, urgency) in cursor.execute( """SELECT DISTINCT sp.name, n.id, sp.subrelease, st.vulnerable, n.urgency FROM source_package_status AS st, source_packages AS sp, package_notes AS n WHERE st.bug_name = ? AND sp.rowid = st.package - AND sp.release = ? AND sp.subrelease IN ('', 'security') + AND sp.release = ? AND sp.subrelease IN ('', 'security', 'lts') AND n.bug_name = st.bug_name AND n.package = sp.name ORDER BY sp.name""", (bug_name, nickname)): @@ -1166,6 +1173,8 @@ unfixed_pkgs[package] = True if status['security'].get((package, note), True): fixed_in_security = False + elif status['lts'].get((package, note), True): + fixed_in_security = False elif vulnerable == 2: undet_pkgs[package] = True @@ -1277,7 +1286,7 @@ FROM source_packages AS p, source_package_status AS st WHERE p.name = ? AND p.release = ? - AND p.subrelease IN ('', 'security') + AND p.subrelease IN ('', 'security', 'lts') AND st.bug_name = ? AND st.package = p.rowid ORDER BY p.version COLLATE version DESC""" @@ -1438,10 +1447,10 @@ # covers binary-only NMUs. for (v,) in c.execute("""SELECT version FROM source_packages WHERE name = ?1 - AND release = ?2 AND subrelease IN ('', 'security') + AND release = ?2 AND subrelease IN ('', 'security', 'lts') UNION ALL SELECT source_version FROM binary_packages WHERE source = ?1 - AND release = ?2 AND subrelease IN ('', 'security')""", + AND release = ?2 AND subrelease IN ('', 'security', 'lts')""", (package, release)): if debian_support.Version(v) >= v_ref: other_versions[v] = True @@ -1660,14 +1669,14 @@ AND COALESCE((SELECT st2.vulnerable FROM source_packages AS sp2, source_package_status AS st2 WHERE sp2.name = sp.name AND sp2.release = sp.release - AND sp2.subrelease = 'security' AND sp2.archive = sp.archive + AND ( sp2.subrelease = 'security' OR sp2.subrelease = 'lts' ) AND sp2.archive = sp.archive AND st2.package = sp2.rowid AND st2.bug_name = st.bug_name ORDER BY st2.vulnerable DESC), 1)) AS vulnerable, st.urgency = 'unimportant' OR NOT vulnerable AS unimportant FROM source_packages AS sp, source_package_status AS st, bugs WHERE sp.name = ? AND sp.release IN ('squeeze', 'wheezy', 'jessie', 'sid') - AND sp.subrelease <> 'security' + AND sp.subrelease <> 'security' AND p.subrelease <> 'lts' AND st.package = sp.rowid AND bugs.name = st.bug_name AND bugs.name NOT LIKE 'DSA-%' @@ -1680,9 +1689,10 @@ """SELECT bugs.name, bugs.description FROM bugs, package_notes as p WHERE p.bug_name = bugs.name - AND bugs.name LIKE 'DSA-%' + AND ( bugs.name LIKE 'DSA-%' OR bugs.name LIKE 'DLA-%') AND p.package = ?""", (package,)) + def getTODOs(self, cursor=None, hide_check=False): """Returns a list of pairs (BUG-NAME, DESCRIPTION).""" if cursor is None: @@ -1928,6 +1938,7 @@ assert not b.not_for_us assert 'DSA-800-1' in b.xref, b.xref assert 'DTSA-10-1' in b.xref, b.xref + assert 'DLA-23-1' in b.xref, b.xref assert tuple(b.comments) == (('NOTE', 'gnumeric/goffice includes one as well; according to upstream not exploitable in gnumeric,'), ('NOTE', 'new copy will be included any way')),\ b.comments Index: lib/python/sectracker_test/test_parsers.py =================================================================== --- lib/python/sectracker_test/test_parsers.py (Revision 28502) +++ lib/python/sectracker_test/test_parsers.py (Arbeitskopie) @@ -40,6 +40,11 @@ for err in o.messages: print "%s:%d: %s: %s" % (err.file, err.line, err.level, err.message) +safeunlink("../../data/DLA/list" + EXTENSION) +o = dlalist("../../data/DLA/list") +for err in o.messages: + print "%s:%d: %s: %s" % (err.file, err.line, err.level, err.message) + Message = sectracker.diagnostics.Message for (line, res, xmsgs) in [ (' - foo <unfixed>', Index: lib/python/sectracker_test/test_analyzers.py =================================================================== --- lib/python/sectracker_test/test_analyzers.py (Revision 28502) +++ lib/python/sectracker_test/test_analyzers.py (Arbeitskopie) @@ -26,6 +26,7 @@ diag = Diagnostics() bugdb = mergelists((p.cvelist("../../data/CVE/list"), p.dsalist("../../data/DSA/list"), + p.dlalist("../../data/DLA/list"), p.dtsalist("../../data/DTSA/list")), diag) assert "CVE-1999-0001" in bugdb assert "DSA-135" in bugdb Index: lib/python/sectracker/parsers.py =================================================================== --- lib/python/sectracker/parsers.py (Revision 28502) +++ lib/python/sectracker/parsers.py (Arbeitskopie) @@ -313,3 +313,20 @@ _checkrelease(anns, diag, "DTSA") return Bug(path, Header(headerlineno, name, None), tuple(anns)) return _parselist(path, f, parseheader, finish) + +@_xpickle.loader("DLA" + FORMAT) +def dlalist(path, f): + re_header = re.compile( + r'^\[([A-Z][a-z]{2,}) (\d\d?)(?:st|nd|rd|th), (\d{4})\] ' + + r'(DLA-\d+-\d+)\s+' + + r'(.*?)\s*$') + def parseheader(line): + match = re_header.match(line) + if match is None: + return None + return match.groups() + def finish(header, headerlineno, anns, diag): + d, m, y, name, desc = header + _checkrelease(anns, diag, "DLA") + return Bug(path, Header(headerlineno, name, None), tuple(anns)) + return _parselist(path, f, parseheader, finish) Index: lib/python/bugs.py =================================================================== --- lib/python/bugs.py (Revision 28502) +++ lib/python/bugs.py (Arbeitskopie) @@ -418,9 +418,9 @@ re_whitespace = re.compile(r'\s+') re_xref_entry = re.compile('^(?:CVE-\d{4}-\d{4,}' + r'|VU#\d{6}' - + r'|DSA-\d+(?:-\d+)?|DTSA-\d+-\d+)$') + + r'|DSA-\d+(?:-\d+)?|DTSA-\d+-\d+|DLA-\d+-\d+)$') re_xref_entry_own = re.compile( - '^(?:CVE-\d{4}-\d{4,}|DSA-\d+(?:-\d+)?|DTSA-\d+-\d+)$') + '^(?:CVE-\d{4}-\d{4,}|DSA-\d+(?:-\d+)?|DTSA-\d+-\d+|DLA-\d+-\d+)$') re_package_required = re.compile(r'^(?:\[.*\]\s*)?-') re_package_version = re.compile( @@ -808,7 +808,48 @@ # Merge identical package notes, for historical reasons. bug.mergeNotes() return bug - + +class DLAFile(FileBase): + """A DLA file. + + Similar to a CVE file, only that it contains DLAs as its main + reference point, and release dates. + """ + + re_dsa = re.compile(r'^\[(\d\d) ([A-Z][a-z][a-z]) (\d{4})\] ' + + r'(DLA-\d+(?:-\d+)?)\s+' + + r'(.*?)\s*$') + + month_names = {'Jan': 1, + 'Feb': 2, + 'Mar': 3, + 'Apr': 4, + 'May': 5, + 'Jun': 6, + 'Jul': 7, + 'Aug': 8, + 'Sep': 9, + 'Oct': 10, + 'Nov': 11, + 'Dec': 12} + + def matchHeader(self, line): + match = self.re_dsa.match(line) + if not match: + self.raiseSyntaxError("expected DLA record, got: %s" % `line`) + (record_name, description) = match.groups() + (day, month, year, name, desc) = match.groups() + try: + month = self.month_names[month] + except KeyError: + self.raiseSyntaxError("invalid month name %s" % `month`) + return ("%s-%02d-%s" % (year, month, day), name, desc) + + def finishBug(self, bug): + # Merge identical package notes, for historical reasons. + bug.mergeNotes() + return bug + class DTSAFile(FileBase): """A DTSA file. Index: bin/updatelist =================================================================== --- bin/updatelist (Revision 28502) +++ bin/updatelist (Arbeitskopie) @@ -2,6 +2,7 @@ my $html=shift; my $dsa_list=shift; my $dtsa_list=shift; +my $dla_list=shift; my $our_list=shift; my %cves; @@ -28,6 +29,7 @@ } read_dsa($dsa_list); read_dsa($dtsa_list); +read_dsa($dla_list); my %listedcves; Index: bin/update =================================================================== --- bin/update (Revision 28502) +++ bin/update (Arbeitskopie) @@ -10,5 +10,5 @@ rm -f allitems.html wget --quiet https://cve.mitre.org/data/downloads/allitems.html.gz gunzip allitems.html.gz -../../bin/updatelist allitems.html ../DSA/list ../DTSA/list list > list.new +../../bin/updatelist allitems.html ../DSA/list ../DTSA/list ../DLA/list list > list.new mv -f list.new list Index: bin/check-syntax =================================================================== --- bin/check-syntax (Revision 28502) +++ bin/check-syntax (Arbeitskopie) @@ -65,9 +65,13 @@ def parse_DTSA(name): do_parse(construct(bugs.DTSAFile, name)) +def parse_DLA(name): + do_parse(construct(bugs.DLAFile, name)) + file_types = {'CVE' : parse_CVE, 'DSA' : parse_DSA, - 'DTSA' : parse_DTSA} + 'DTSA' : parse_DTSA, + 'DLA' : parse_DLA} if len(sys.argv) <> 3 or not file_types.has_key(sys.argv[1]): l = file_types.keys() Index: bin/tracker_service.py =================================================================== --- bin/tracker_service.py (Revision 28502) +++ bin/tracker_service.py (Arbeitskopie) @@ -342,6 +342,8 @@ source_xref = self.make_dsa_ref(url, bug.name, 'Debian') elif source == 'DTSA': source_xref = 'Debian Testing Security Team' + elif source == 'DLA': + source_xref = 'Debian LTS Team' elif source == 'TEMP': source_xref = ( 'Automatically generated temporary name. Not for external reference.')signature.asc
Description: This is a digitally signed message part.
--- End Message ---
--- Begin Message ---* Holger Levsen: >> I believe the fix is this additional change in getBugsForSourcePackage: >> >> AND bugs.name = st.bug_name >> - AND bugs.name NOT LIKE 'DSA-%' >> + AND bugs.name LIKE 'CVE-%' > > right, cool! maybe LIKE ('CVE-%' OR 'TEMP-%') ?? Or are those really never > used? That's indeed much better. I've made this additional change. The code is now running on soler, so I'm closing this bug.
--- End Message ---