Bug#908678: Update on the security-tracker git discussion
Hi Salvatore,
On Sat, Jun 08, 2019 at 06:29:24PM +0200, Salvatore Bonaccorso wrote:
> Hi,
>
> On Thu, Jun 06, 2019 at 06:11:53PM +0200, Salvatore Bonaccorso wrote:
> > Hi Daniel,
> >
> > On Thu, Jun 06, 2019 at 08:35:47AM +0200, Daniel Lange wrote:
> > > Am 06.06.19 um 07:31 schrieb Salvatore Bonaccorso:
> > > > Could you again point me to your splitted up variant mirror?
> > >
> > > https://git.faster-it.de/debian_security_security-tracker_split_files/
> >
> > Thanks!
> >
> > While starting to look at it, could you change the splitting to
> > $year.list instead of list.$year? I know this comes from the initial
> > script which was commited. It is though more intuitive working with
> > $work.something than something.$year in this context.
>
> Thanks to Daniel for providing the converted repository (with list
> named as well the other way around as $year.list, which is more
> intuitive, and looks saner (to me)) which get updated regularly, this
> helps as a extremly good basis.
>
> Below are some thoughs which I started thinking of during the last few
> days, please not it might not yet be complete. Please as well try to
> not push/force us too much -- whilst we understand the issue, and see
> that something whatever the solution is (split, move somewhere else)
> -- we have regularly more serious issues popping up we want and need
> to look at those. But we acknowledge and see als well salsa admin
> point of view.
>
> That said, here is what I have at the moment, some are easy, some
> will/might be more involving.
>
> Notes on possible CVE/list splits
> -
>
> - workflows on files itself by most active users. Often kept open
> cross-checking issues all issues in one file. But this will "just"
> need other ways to deal with the situation by the persons working
> most on it.
> - Code of security-tracker service and python modules itself which
> currently rely on the data/*/list formats (DSA, DLA, CVE, ...) This
> could probably be split up and use data/*/*.list
> - Externally called but included in code: update script which fetches
> MITRE list and integrates all needed changes (see further below).
> - bin/bts-update (called from scripts/update-CVE-assignments in cron of
> the securiy-tracker-services) operates based on data/CVE/list and
> keeps track of the already tagged bugs by comparing with an 'oldlist'.
> The oldlist is copied on a run on soriano.debian.org as 'state' file
> similar to logroate's statefile (cron).
> - bin/check-new-issues: parsing of TODO and checks for the new issues is
> as well based on 'data/CVE/list' existence and parsing. After a split
> up the interactive commands should still be able to navigate trough
> the items.
> - bin/check-syntax: Check syntax of the various lists based on the security-
> tracker parser for the lists. make check-syntax from the Makefile, pre-
> commit hook or C/I tests are all using this script for syntax check.
> Depends on CVEfile as well from python/bugs.py. Relevant here is the
> check-syntax target from the Makefile. At SVN times this was actually
> only testing the syntax of the changed files, but now it just runs
> make check-syntax.
> - bin/compare-nvd-cve reads from data/CVE/list and this is probably
> easier to adapt and it's used basically in a "experimental" target in
> Makefile for update-compare-nvd target. AFAICS this is just reading
> the information should be easy to adapt to any split up setup.
> - bin/gen-{DSA,DLA}: Used the data/CVE/list for sanity check for
> presence of the CVE.
> - bin/get-todo-items (this script is currently not working correctly and
> it's implemented already via the webview, so need to consider if we
> actually still need it).
> - bin/inject-embedded-code-copies (experimental script, not
> actively used)
> - bin/rejected-with-info relies on data/CVE/list directly, but will be
> potentially easily adaptable in a splited setup.
> - bin/setup-repo: checks for data/CVE/list just to make sure it's the
> right repo.
> - bin/report-vuln uses CVEFile (from python/bugs.py).
> - bin/update and bin/updatelist: Parses DSA/DTSA/DLA list and
> data/CVE/list adding new entries from MITRE feed and crossreferences
> for the DSA/DLA's to a new data/CVE/list which then in the cronjob on
> soriano will be committed. That is one processing those files in a
> splitted setup this will need continue to work.
> - bin/update-db (Used triggered by Makefile target to update security.db
> sqlite database).
> - bin/update-nvd (possibly dependency on the CVE lists via the used
> modules but not directly).
> - data/config.json contains the sources for CVE, DSA, DLA and extended
> lists. Currently path thus will be a path component starting from
> data, e.g. for CVE files path is '/CVE/list'. See as well "Setting up
> an extended instance" in the documentation.
> - lib/python/bugs.py contains the classes CVEFile, DSAFile,
>
Bug#908678: Testing the filter-branch scripts
Hi, On Tue, Nov 13, 2018 at 11:09:41PM +0100, Moritz Muehlenhoff wrote: > On Tue, Nov 13, 2018 at 12:22:54PM -0500, Antoine Beaupré wrote: > > But before going through that trouble, I think we'd need to get approval > > from the security team first, as that's quite a lot of work. I figured > > we would make a feasability study first... > > The current data structure works very well for us and splitting the files > has many downsides. > > If we can't get the repository in run on salsa in a manner that doesn't > impact other repositories (e.g. by disabling the repository browser or > similar), then moving the security tracker repository out of Salsa is > the more likely solution. > > Did anyone follow Guido's suggestion to report this upstream to > get their assessment on possible optimisations? Just in case someone takes this upstream. I've filed https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=913124 against git a couple of days ago. Cheers, -- Guido
Bug#908678: Some more thoughts and some tests on the security-tracker git repo
Hi, On Wed, Sep 26, 2018 at 01:56:16PM +0200, Daniel Lange wrote: > The main issue is that we need to get clone and diff+render operations > back into normal time frames. The salsa workers (e.g. to render a > diff) time out after 60s. Similar time constraints are put onto other I wonder why that is since "git diff" is pretty fast on a local checkout. Did we ask the gitlab folks about it? [..snip..] > Just splitting the file will not do. We need to (unfortunately) > somehow "get rid" of the history (delta-resolution) walks in git: Not necessarily. Maybe a graft would do: https://developer.atlassian.com/blog/2015/08/grafting-earlier-history-with-git/ This is IMHO preferable over history rewrites. I've used this to tie histories in the past. I've not used "git replace" though but .git/info/grafts. Cheers, -- Guido
Bug#908678: security-tracker - Breaks salsa.d.o
Hi, On Tue, Sep 25, 2018 at 09:00:49PM +0200, Salvatore Bonaccorso wrote: > One suggestion from IRC discussion: > > < DLange> summary: suggestions are along the idea of creating list-$year and > combine in list for current tools or amend them? I think that makes sense. An alternative would be to use shallow clones (--depth=1) on clones for all the tools (and to recommend it in the docs). Did somebody contact git upstream yet? It might be worth showing this use case. Cheers, -- Guido
Re: DST down ?
Hi Fried, On Fri, Dec 29, 2017 at 08:30:47PM +0100, Fried Wil wrote: > Hi DST Team, > > How are you ? Hope you enjoyed and have some good time for Xmas. > > Is DST website in maintenance or something ? There is no list on all > releases [1] > > And I test some DST : not working [2][3] ; always "Not found". Please see https://salsa.debian.org/security-tracker-team/security-tracker Content is available again already. Cheers, -- Guido > > Thanks in advance. > > Regards, > > [1] https://security-tracker.debian.org/tracker/status/release/stable > [2] https://security-tracker.debian.org/tracker/wireshark > [3] https://security-tracker.debian.org/tracker/CVE-2017-17934 > > -- > Wilfried Pascault > wilfried.pasca...@gmail.com >
Re: Splitting the security-tracker repo [was Re: [PATCH 00/12] Plannings for secure-testing repository migration to git]
Hi Salvatore, On Thu, Dec 28, 2017 at 09:02:56PM +0100, Salvatore Bonaccorso wrote: > Hi Guido, > > On Thu, Dec 28, 2017 at 06:59:11PM +0100, Guido Günther wrote: > > Hi, > > On Thu, Dec 28, 2017 at 12:33:27PM +0100, Guido Günther wrote: > > > Hi, > > > On Wed, Dec 27, 2017 at 11:32:53PM +0100, Salvatore Bonaccorso wrote: > > > [..snip..] > > > > have that already :). But I'm unahappy with my current versions, thus > > > > I have not sent those yet. Basically: I'm struggling with a proper > > > > replacement of such constructs: > > > > > > > > cd /srv/security-tracker.debian.org/website/secure-testing/data && svn > > > > update && ../bin/update && svn commit -qm "automatic update" CVE && cd > > > > CVE && ../../bin/bts-update list > > > > > >cd /srv/security-tracker.debian.org/website/security-tracker/data && > > > git pull origin master && ../bin/update && git commit -qm "automatic > > > update" CVE && git push origin master && cd CVE && ../../bin/bts-update > > > list > > > > > > git commit might fail if there are no changes so we need to either guard > > > for that or use --allow-empty. git push might fail as well if there were > > > pushes in between. Since there are several commands that > > > > > > pull, modify, commit push > > > > > > should we move this to a script that > > > > > > pulls, invokes the modification command, commits when there are > > > changes, pushes. If that fails merges again && pushes again. > > > > > > That won't save you from merge conflicts but that's the same with SVN a > > > assume? > > > > Salvatore pointed out to me that it would be good to not mix tracker > > code updates with data updates so that the above commands that pull and > > push data to/from git would only operate on the data part but not on the > > tracker itself. One possible solution would be to split the current > > secure-testing repository into two: > > > > security-tracker: containing the data that is the data/ packages/ org/ > > doc/ and conf/ directories. > > security-tracker-bin: containing the security tracker service and other > > scripts and python modules: check-external, bin, tools, lib, > > templates, website > > > > security-tracker-bin could then become a submodule of security > > tracker. To ease migration we would create symlinks so the directory > > layout stays the same. The top level of the secure-testing repo would > > look something like this: > > > > -rw-r--r-- 1 agx agx 3449 Dez 28 11:56 TODO.gitmigration > > drwxr-sr-x 9 agx agx 4096 Dez 28 11:56 data > > drwxr-sr-x 2 agx agx 4096 Dez 28 11:56 packages > > drwxr-sr-x 2 agx agx 4096 Dez 28 11:56 org > > drwxr-sr-x 5 agx agx 4096 Dez 28 11:56 doc > > -rw-r--r-- 1 agx agx 8450 Dez 28 12:01 Makefile > > drwxr-sr-x 2 agx agx 4096 Dez 28 13:30 conf > > -rw-r--r-- 1 agx agx 179 Dez 28 17:20 .gitmodules > > lrwxrwxrwx 1 agx agx 35 Dez 28 17:20 check-external -> > > security-tracker-bin/check-external > > lrwxrwxrwx 1 agx agx 24 Dez 28 17:20 bin -> security-tracker-bin/bin > > lrwxrwxrwx 1 agx agx 26 Dez 28 17:20 tools -> > > security-tracker-bin/tools > > lrwxrwxrwx 1 agx agx 30 Dez 28 17:20 templates -> > > security-tracker-bin/templates > > lrwxrwxrwx 1 agx agx 27 Dez 28 17:20 static -> > > security-tracker-bin/static > > lrwxrwxrwx 1 agx agx 24 Dez 28 17:20 lib -> security-tracker-bin/lib > > lrwxrwxrwx 1 agx agx 28 Dez 28 17:20 website -> > > security-tracker-bin/website > > drwxr-sr-x 10 agx agx 4096 Dez 28 17:20 . > > drwxr-sr-x 9 agx agx 4096 Dez 28 17:25 security-tracker-bin > > drwxr-sr-x 2 agx agx 4096 Dez 28 17:26 stamps > > drwxr-sr-x 9 agx agx 4096 Dez 28 17:26 .git > > drwxr-sr-x 4 agx agx 4096 Dez 28 18:17 .. > > > > I have scripts¹ that produce such a layout and pushed temporary repos > > here > > That looks good to me, can you push your conversion scripts to > tools/git-migration? Comitted to the security-testing svn repository. Cheers, -- Guido
Splitting the security-tracker repo [was Re: [PATCH 00/12] Plannings for secure-testing repository migration to git]
Hi, On Thu, Dec 28, 2017 at 12:33:27PM +0100, Guido Günther wrote: > Hi, > On Wed, Dec 27, 2017 at 11:32:53PM +0100, Salvatore Bonaccorso wrote: > [..snip..] > > have that already :). But I'm unahappy with my current versions, thus > > I have not sent those yet. Basically: I'm struggling with a proper > > replacement of such constructs: > > > > cd /srv/security-tracker.debian.org/website/secure-testing/data && svn > > update && ../bin/update && svn commit -qm "automatic update" CVE && cd CVE > > && ../../bin/bts-update list > >cd /srv/security-tracker.debian.org/website/security-tracker/data && git > pull origin master && ../bin/update && git commit -qm "automatic update" CVE > && git push origin master && cd CVE && ../../bin/bts-update list > > git commit might fail if there are no changes so we need to either guard > for that or use --allow-empty. git push might fail as well if there were > pushes in between. Since there are several commands that > > pull, modify, commit push > > should we move this to a script that > > pulls, invokes the modification command, commits when there are > changes, pushes. If that fails merges again && pushes again. > > That won't save you from merge conflicts but that's the same with SVN a > assume? Salvatore pointed out to me that it would be good to not mix tracker code updates with data updates so that the above commands that pull and push data to/from git would only operate on the data part but not on the tracker itself. One possible solution would be to split the current secure-testing repository into two: security-tracker: containing the data that is the data/ packages/ org/ doc/ and conf/ directories. security-tracker-bin: containing the security tracker service and other scripts and python modules: check-external, bin, tools, lib, templates, website security-tracker-bin could then become a submodule of security tracker. To ease migration we would create symlinks so the directory layout stays the same. The top level of the secure-testing repo would look something like this: -rw-r--r-- 1 agx agx 3449 Dez 28 11:56 TODO.gitmigration drwxr-sr-x 9 agx agx 4096 Dez 28 11:56 data drwxr-sr-x 2 agx agx 4096 Dez 28 11:56 packages drwxr-sr-x 2 agx agx 4096 Dez 28 11:56 org drwxr-sr-x 5 agx agx 4096 Dez 28 11:56 doc -rw-r--r-- 1 agx agx 8450 Dez 28 12:01 Makefile drwxr-sr-x 2 agx agx 4096 Dez 28 13:30 conf -rw-r--r-- 1 agx agx 179 Dez 28 17:20 .gitmodules lrwxrwxrwx 1 agx agx 35 Dez 28 17:20 check-external -> security-tracker-bin/check-external lrwxrwxrwx 1 agx agx 24 Dez 28 17:20 bin -> security-tracker-bin/bin lrwxrwxrwx 1 agx agx 26 Dez 28 17:20 tools -> security-tracker-bin/tools lrwxrwxrwx 1 agx agx 30 Dez 28 17:20 templates -> security-tracker-bin/templates lrwxrwxrwx 1 agx agx 27 Dez 28 17:20 static -> security-tracker-bin/static lrwxrwxrwx 1 agx agx 24 Dez 28 17:20 lib -> security-tracker-bin/lib lrwxrwxrwx 1 agx agx 28 Dez 28 17:20 website -> security-tracker-bin/website drwxr-sr-x 10 agx agx 4096 Dez 28 17:20 . drwxr-sr-x 9 agx agx 4096 Dez 28 17:25 security-tracker-bin drwxr-sr-x 2 agx agx 4096 Dez 28 17:26 stamps drwxr-sr-x 9 agx agx 4096 Dez 28 17:26 .git drwxr-sr-x 4 agx agx 4096 Dez 28 18:17 .. I have scripts¹ that produce such a layout and pushed temporary repos here https://salsa.debian.org/agx/security-tracker-with-submodule https://salsa.debian.org/agx/security-tracker-bin So if you want to see how it looks to a git clone --recursive https://salsa.debian.org/agx/security-tracker-with-submodule With some follow up work we can get rid of most of the symlinks but this could happen after the git migration. So soriano would only see an update of security-tracker-bin code if we bump the submodule revision (end even then only if we do a "git submodule update"). Does this make sense or does it introduce too much complexity? Is the above split about right or should some of the directories move between security-tracker and security-tracker-bin (e.g. website or static). Feedback would be welcome. Cheers, -- Guido ¹: https://salsa.debian.org/agx/security-tracker/commit/4a4a83117e3d6791fbe091a4de2a2bc1ad0abf98
Re: [PATCH 00/12] Plannings for secure-testing repository migration to git
Hi,
On Thu, Dec 28, 2017 at 12:45:38PM +0100, Guido Günther wrote:
[..snip..]
> We could provide a pre-commit hook:
>
> https://git-scm.com/book/gr/v2/Customizing-Git-Git-Hooks
>
> they're not enforced though. User would have to set it up via
>
> ln -s bin/pre-commit .git/hooks/pre-commit
>
> (which we could put into a "bin/setup-repo"). Since the hook can
> identify changed files we could only run tests on changed files so it
> would be reasonably fast. I'll have a look.
What about adding this to the secure-testing-repo (the makefile already
creates stamps so we don't need to differentiate in the hook again). We
might add other stuff like installing dependencies to run the various
commands to setup-repo later on.
>From 9e0f77e41b4e194e320609c0d74ced3a82b06f05 Mon Sep 17 00:00:00 2001
Message-Id:
<9e0f77e41b4e194e320609c0d74ced3a82b06f05.1514463957.git@sigxcpu.org>
From: =?UTF-8?q?Guido=20G=C3=BCnther?= <a...@sigxcpu.org>
Date: Thu, 28 Dec 2017 13:05:05 +0100
Subject: [PATCH] Add pre-commit hook to check syntax
---
bin/setup-repo | 31 +++
conf/pre-commit | 12
2 files changed, 43 insertions(+)
create mode 100755 bin/setup-repo
create mode 100755 conf/pre-commit
diff --git a/bin/setup-repo b/bin/setup-repo
new file mode 100755
index 00..ef7cf2c95b
--- /dev/null
+++ b/bin/setup-repo
@@ -0,0 +1,31 @@
+#!/bin/sh
+#
+# Set up a clone of the security-tracker git repo
+
+set -e
+
+SRC=../../conf/pre-commit
+HOOK=.git/hooks/pre-commit
+
+install_pre_commit_hook() {
+ if [ -L "${HOOK}" ] && [ "$(readlink ${HOOK})" = "${SRC}" ]; then
+ echo "pre-commit hook already set up"
+ return
+ fi
+
+ if [ -e "${HOOK}" ] || [ -L "${HOOK}" ]; then
+echo "Moving old pre-commit hook"
+ mv -f "${HOOK}" "${HOOK}.$(date '+%s')"
+ fi
+
+ echo "Installing pre-commit hook"
+ ln -s "${SRC}" "${HOOK}"
+}
+
+
+if [ "$(git rev-parse --show-cdup)" != '' ] || [ -d data/CVE/list ]; then
+ echo "This does not look like the git repo of the security tracker" 1>&2
+ exit 1
+fi
+
+install_pre_commit_hook
diff --git a/conf/pre-commit b/conf/pre-commit
new file mode 100755
index 00..f097a88abd
--- /dev/null
+++ b/conf/pre-commit
@@ -0,0 +1,12 @@
+#!/bin/sh
+
+set -e
+
+if [ -z "${GIT_DIR}" ]; then
+echo "GIT_DIR not set" 1>&2
+exit 1
+fi
+
+exec 1>&2
+
+make check-syntax
--
2.15.1
Re: [PATCH 00/12] Plannings for secure-testing repository migration to git
Hi, On Thu, Dec 28, 2017 at 09:13:09AM +0100, Salvatore Bonaccorso wrote: > Hi > > On Wed, Dec 27, 2017 at 11:32:53PM +0100, Salvatore Bonaccorso wrote: > > 1/ desire of commit mailinglist > > Would there be objection to switch those to > debian-security-tracker@l.d.o? A dedicated list IHMO would be better > to distinguish between the commit review list and the general > discussion/bug list. > > > 2/ how to do the updates with the sectracker role account specific to > > some files (e.g. automatic updates, data/CVE/list update, dsa > > canditate finding). > > Btw, 3/ is there a way to force a pre-commit/pre-push action with git > repositories (so client side hooks forced on every cloned repository) > which would run 'make check-syntax' before we push something? For the > subversion repository this was implemented with a pre-commit svn hook, > which called for changes with bin/check-syntax, or > bin/check-new-issues -c (for the data/embedded-code-copies file). > > I guess the short answer is 'no'. We could provide a pre-commit hook: https://git-scm.com/book/gr/v2/Customizing-Git-Git-Hooks they're not enforced though. User would have to set it up via ln -s bin/pre-commit .git/hooks/pre-commit (which we could put into a "bin/setup-repo"). Since the hook can identify changed files we could only run tests on changed files so it would be reasonably fast. I'll have a look. > I got the reply for Salsa, for the custom_hooks referring to > https://docs.gitlab.com/ce/administration/custom_hooks.html but as far > I understand those are server-side running hooks. Will it be possible for us to add such a hook on salsa? AFAIK adding these needs admin permissions on the GitLab instance. Having an in-flight syntax-check is a nice feature of the current setup which would be good to preserve. While we could set up a something that tests every merge request using merge requests looks like overkill here. Cheers, -- Guido
Re: [PATCH 00/12] Plannings for secure-testing repository migration to git
Hi, On Wed, Dec 27, 2017 at 11:32:53PM +0100, Salvatore Bonaccorso wrote: [..snip..] > have that already :). But I'm unahappy with my current versions, thus > I have not sent those yet. Basically: I'm struggling with a proper > replacement of such constructs: > > cd /srv/security-tracker.debian.org/website/secure-testing/data && svn update > && ../bin/update && svn commit -qm "automatic update" CVE && cd CVE && > ../../bin/bts-update list cd /srv/security-tracker.debian.org/website/security-tracker/data && git pull origin master && ../bin/update && git commit -qm "automatic update" CVE && git push origin master && cd CVE && ../../bin/bts-update list git commit might fail if there are no changes so we need to either guard for that or use --allow-empty. git push might fail as well if there were pushes in between. Since there are several commands that pull, modify, commit push should we move this to a script that pulls, invokes the modification command, commits when there are changes, pushes. If that fails merges again && pushes again. That won't save you from merge conflicts but that's the same with SVN a assume? Cheers, -- Guido > > Should we create WiP/git-migration branches in the secure-testing and > > security-tracker-service repos so others can send in pull requests? > > > > We could rebase the branch in the secure-testing repo from time to time > > so catch up with the changes in the svn repo. > > Give me a bit of time to think about it. My intention was to not > clutter now what we have on salsa as mirror and constantly push > updates there. And once we go live, add the Debian group as allowed to > push, and block the original svn repository. I would like to avoid > that people already start pushing to the git repository directly. > > Or do you mean to still create the branches on the git repo on salsa > (accessible readonly, so that people can contribute based on that, > rather based on patch for pull requests? As said, I would like to not > operate directly on the git repo until the switch to not cause > problems. Our live instance is currently still the svn repo). > > I would expect we solve the open problems soonish enough so we can do > the switch. That is: > > 1/ desire of commit mailinglist > 2/ how to do the updates with the sectracker role account specific to > some files (e.g. automatic updates, data/CVE/list update, dsa > canditate finding). > > Thanks a lot for your contribution Guido! > > Salvatore >
Re: [PATCH 00/12] Plannings for secure-testing repository migration to git
Hi Salvatore,
On Wed, Dec 27, 2017 at 11:32:53PM +0100, Salvatore Bonaccorso wrote:
> Hi Guido
>
> Much appreciated you looked at the patchset!
>
> On Wed, Dec 27, 2017 at 09:14:52PM +0100, Guido Günther wrote:
> > Hi,
> > On Wed, Dec 27, 2017 at 08:42:49PM +0100, Salvatore Bonaccorso wrote:
> > > This *preliminary* patchset for review is for the changes needed after
> > > we would switch from alioth to salsa for the secure-testing repository.
> > > While needing to do the svn to git conversion the opportunity was taken
> > > to rename the repository to security-tracker as the naming
> > > secure-testing was for historical reason when the testing security
> > > support arised.
> >
> > This looks good! I applied your patches to my checkout I found some more
> > svn references in
> >
> > * tracker 'latest_revision' method
>
> Jupp that was what I mean that I need here some insights from a LTS
> member and how the script functions.
Here's a patch also available at
https://salsa.debian.org/agx/security-tracker/commit/805a2b5fe27f9812dc679c231ab4dd1a1c9613c1
>From 81025922f7037c4e3b86ac1aff9cd15432f42409 Mon Sep 17 00:00:00 2001
Message-Id:
<81025922f7037c4e3b86ac1aff9cd15432f42409.1514457764.git@sigxcpu.org>
From: =?UTF-8?q?Guido=20G=C3=BCnther?= <a...@sigxcpu.org>
Date: Thu, 28 Dec 2017 11:36:55 +0100
Subject: [PATCH] TrackerData: use git in instead of svn
Use git-ls-remote instead of svn-info to determine the head revision.
---
bin/tracker_data.py | 15 ---
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/bin/tracker_data.py b/bin/tracker_data.py
index 3bbac16603..3aaefbe707 100644
--- a/bin/tracker_data.py
+++ b/bin/tracker_data.py
@@ -45,14 +45,15 @@ def normalize_release(release):
class TrackerData(object):
DATA_URL = "https://security-tracker.debian.org/tracker/data/json;
+GIT_URL =
"https://salsa.debian.org/security-tracker-team/security-tracker-service.git;
CACHED_DATA_PATH = "~/.cache/debian_security_tracker.json"
CACHED_REVISION_PATH = "~/.cache/debian_security_tracker.rev"
GET_REVISION_COMMAND = \
-"LC_ALL=C svn info svn://anonscm.debian.org/secure-testing|"\
-"awk '/^Revision:/ { print $2 }'"
+"LC_ALL=C git ls-remote %s | awk '/HEAD$/ { print $1 }'" % GIT_URL
DATA_DIR = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'data')
def __init__(self, update_cache=True):
+self._latest_revision = None
self.cached_data_path = os.path.expanduser(self.CACHED_DATA_PATH)
self.cached_revision_path = os.path.expanduser(
self.CACHED_REVISION_PATH)
@@ -62,14 +63,14 @@ class TrackerData(object):
@property
def latest_revision(self):
-"""Return the current revision of the SVN repository"""
+"""Return the current revision of the Git repository"""
# Return cached value if available
-if hasattr(self, '_latest_revision'):
+if self._latest_revision is not None:
return self._latest_revision
-# Otherwise call out to svn to get the latest revision
+# Otherwise call out to git to get the latest revision
output = subprocess.check_output(self.GET_REVISION_COMMAND,
shell=True)
-self._latest_revision = int(output)
+self._latest_revision = output.strip()
return self._latest_revision
def _cache_must_be_updated(self):
@@ -78,7 +79,7 @@ class TrackerData(object):
self.cached_revision_path):
with open(self.cached_revision_path, 'r') as f:
try:
-revision = int(f.readline())
+revision = f.read()
except ValueError:
revision = None
if revision == self.latest_revision:
--
2.15.1
Re: [PATCH 00/12] Plannings for secure-testing repository migration to git
Hi, On Wed, Dec 27, 2017 at 08:42:49PM +0100, Salvatore Bonaccorso wrote: > This *preliminary* patchset for review is for the changes needed after > we would switch from alioth to salsa for the secure-testing repository. > While needing to do the svn to git conversion the opportunity was taken > to rename the repository to security-tracker as the naming > secure-testing was for historical reason when the testing security > support arised. This looks good! I applied your patches to my checkout I found some more svn references in * tracker 'latest_revision' method * website/uploading.html * doc/soriano.txt (which needs to match what we do in security-tracker-service I guess) * the security-tracker-service repo Should we create WiP/git-migration branches in the secure-testing and security-tracker-service repos so others can send in pull requests? We could rebase the branch in the secure-testing repo from time to time so catch up with the changes in the svn repo. Thanks a lot for working on this! -- Guido
Re: samba4 package didn't bundle Heimdal
Hi Andrew, On Thu, Jul 13, 2017 at 09:17:57PM +1200, Andrew Bartlett wrote: > https://security-tracker.debian.org/tracker/CVE-2017-11103 > > Back when samba4 (which has been eviscerated to a client) was a > package, it linked against the system heimdal. > > You can see this because it depends on heimdal. > > https://packages.debian.org/wheezy/libsamba-credentials0 > > Additionally, the link the heimdal code has always been dynamic, not > static, it just changed from dynamic to the system libs to dynamic to > the vendored lib embedded in our tree with the Samba 4.2 packages. Thanks for having a look! I just double checked and indeed the build logs have: [..snip..] Checking for program krb5-config.heimdal : /usr/bin/krb5-config.heimdal ... Selected system Heimdal build [..snip..] There is some stuff compiled from heimdal ... [ 147/2938] Compiling source4/heimdal/lib/vers/print_version.c [ 148/2938] Compiling source4/heimdal_build/version.c [ 149/2938] Compiling source4/heimdal/lib/vers/print_version.c [ 150/2938] Compiling source4/heimdal_build/version.c [ 151/2938] Compiling source4/heimdal/lib/asn1/main.c [ 152/2938] Compiling source4/heimdal/lib/asn1/gen.c [ 153/2938] Compiling source4/heimdal/lib/asn1/gen_copy.c [ 154/2938] Compiling source4/heimdal/lib/asn1/gen_decode.c [ 155/2938] Compiling source4/heimdal/lib/asn1/gen_encode.c [ 156/2938] Compiling source4/heimdal/lib/asn1/gen_free.c [ 157/2938] Compiling source4/heimdal/lib/asn1/gen_glue.c [ 158/2938] Compiling source4/heimdal/lib/asn1/gen_length.c [ 159/2938] Compiling source4/heimdal/lib/asn1/gen_seq.c [ 160/2938] Compiling source4/heimdal/lib/asn1/gen_template.c [ 161/2938] Compiling source4/heimdal/lib/asn1/hash.c [ 162/2938] Compiling source4/heimdal/lib/asn1/symbol.c [ 163/2938] Compiling source4/heimdal/lib/asn1/asn1parse.c [ 164/2938] Compiling source4/heimdal/lib/asn1/lex.c ... but none of the affected code so I've marked samba4 as not affected. Thanks a lot! -- Guido
