Re: Scalable Debian vulnerability tracking
Hi, you might have a look at apt-dater[1] (part of unstable [2]). It is uses SSH to retrieve package informations from client hosts using public key authentification and uses sudo to call apt-get/sudo. It is a ncurses based CLI, but it has a report function to retrieve distri name version, kernel version (and check if a reboot is required), package versions etc. as a XML file - so you could do anything with the result. This work very well for us to keep around 100 hosts up to date. [1] http://apt-dater.sf.net/ [2] http://packages.debian.org/apt-dater Cheers, Thomas -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Scalable Debian vulnerability tracking [REDUX]
http://debianhelpdesk.com/sysinfo.tgz md5sum 0704c3016a64817f1d2dcee7af3b3fa1 This is not production ready. It is currently in production, but only on my clients sites, and I fix things as they pop up. As I say in the README, we're working on a major rewrite that should make it much more usable to others (one of the goals). Feel free to contact me with any questions and/or comments. I take constructive criticism very, very well, and would really like it if anyone could use this at all. I will likely not have the new version production ready for a couple of months. Paying gigs keep getting in the way. Rod R. W. Rodolico wrote: Give me a couple of days to find a version that is not totally unstable. I'll tar it up, get some brief explanation, and post the URL here. Right now, the best 1.x stuff is wrapped up in a .deb. I have 2.0b in testing (on a few machines), but it is showing some bugs. I don't trust my svn install (haven't tested to see if I set it up correctly). I did not make myself clear. The UI to view the summaries and reports is written in PHP. The engines (the client side data gatherer and the server side parser) are all perl. I don't think I've ever written a command line PHP script yet. The big difference between v1 and v2 is that I've gone to xml for the report from the client to the server, and the UI is being broken up into modules so people can add/remove more easily (should have done that from the beginning). In most cases, I do not consider e-mail to be unreliable. However, there are some cases where e-mail is less than optimum. So, I plan to allow https and ftp in the future. Rod Holger Levsen wrote: Hi Sheldon, this sounds like an interesting project, please keep us posted! On Mittwoch, 7. Januar 2009, Sheldon Hearn wrote: On Wednesday 07 January 2009 00:24:09 R. W. Rodolico wrote: I have a package that we have been working on for a while that might be a good starting point. This is gpl'd, and I would be happy to supply the .deb, the source tree or svn access if you would like to look at it. Suppressing my knee-jerk reaction to PHP, it sounds like you're quite far down the track with this one. :-) sitesummary, as in http://packages.qa.debian.org/s/sitesummary.html might also be interested for you to look at. and it's perl, not php. On a site note, I dont consider mail to be too unreliable here. First, it's actually pretty reliable. Second, just resend the mail the next day / time slot. regards, Holger -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Scalable Debian vulnerability tracking [REDUX]
Hi Sheldon, this sounds like an interesting project, please keep us posted! On Mittwoch, 7. Januar 2009, Sheldon Hearn wrote: On Wednesday 07 January 2009 00:24:09 R. W. Rodolico wrote: I have a package that we have been working on for a while that might be a good starting point. This is gpl'd, and I would be happy to supply the .deb, the source tree or svn access if you would like to look at it. Suppressing my knee-jerk reaction to PHP, it sounds like you're quite far down the track with this one. :-) sitesummary, as in http://packages.qa.debian.org/s/sitesummary.html might also be interested for you to look at. and it's perl, not php. On a site note, I dont consider mail to be too unreliable here. First, it's actually pretty reliable. Second, just resend the mail the next day / time slot. regards, Holger signature.asc Description: This is a digitally signed message part.
Re: Scalable Debian vulnerability tracking [REDUX]
Give me a couple of days to find a version that is not totally unstable. I'll tar it up, get some brief explanation, and post the URL here. Right now, the best 1.x stuff is wrapped up in a .deb. I have 2.0b in testing (on a few machines), but it is showing some bugs. I don't trust my svn install (haven't tested to see if I set it up correctly). I did not make myself clear. The UI to view the summaries and reports is written in PHP. The engines (the client side data gatherer and the server side parser) are all perl. I don't think I've ever written a command line PHP script yet. The big difference between v1 and v2 is that I've gone to xml for the report from the client to the server, and the UI is being broken up into modules so people can add/remove more easily (should have done that from the beginning). In most cases, I do not consider e-mail to be unreliable. However, there are some cases where e-mail is less than optimum. So, I plan to allow https and ftp in the future. Rod Holger Levsen wrote: Hi Sheldon, this sounds like an interesting project, please keep us posted! On Mittwoch, 7. Januar 2009, Sheldon Hearn wrote: On Wednesday 07 January 2009 00:24:09 R. W. Rodolico wrote: I have a package that we have been working on for a while that might be a good starting point. This is gpl'd, and I would be happy to supply the .deb, the source tree or svn access if you would like to look at it. Suppressing my knee-jerk reaction to PHP, it sounds like you're quite far down the track with this one. :-) sitesummary, as in http://packages.qa.debian.org/s/sitesummary.html might also be interested for you to look at. and it's perl, not php. On a site note, I dont consider mail to be too unreliable here. First, it's actually pretty reliable. Second, just resend the mail the next day / time slot. regards, Holger -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Scalable Debian vulnerability tracking
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi folks, I work for an hosting provider, and am looking at how to improve visibility into vulnerability exposure. We have over 800 Debian hosts that we manage fore customers, and will have over 1,000 by the end of this quarter. A major problem we face is that our change distribution mechanism is poor. We're working on that problem, but in the meantime, I'm looking at ways to assert that we are / are not vulnerable to specific issues disclosed by the Debian project. I realize that this isn't the whole game, but it's an huge part of it. First prize is a web application that we can draw reports from (or will push reports to us or whatever), that knows what security issues have been identified and addressed by the Debian project, what versions of packages are installed on all servers, and therefore which packages on which servers should have been upgraded but have not yet been. Yup, basically the output of debsecan --only-fixed --suite etch. But I'd prefer not to use email as the transport mechanism (unreliable), and I'd have to write an aggregator for all those mails, because working through mail from over a thousand servers is error prone. So I imagine a solution involving: * a web service to which servers can basically post their dpkg status, * a feed consumer for DSAs, * a vulnerability resolver that matches these two data sources, * reports to provide a view into the persisted resolver results. I spent a lot of time hunting for such a thing, and haven't found one I'm happy with. I'm not keen on using Nessus for this, for a few reasons: * I don't think it's an hosted application with persisted results (i.e. you have to initiate a full network-wide test every time you want a report). * The Tenable sales people are useless. * I don't think their feeds are worth the money. OpenVAS would address the last two, but I can't recommend it in production on Debian Etch or Lenny yet, and there's still the matter that it's not an hosted application with persisted results. So I'm hoping for one of: * Someone points me to an existing app, I bow my head in shame, admit I have no Google Fu, and avoid reinventing the wheel. * Someone points out how wrong I am about Nessus and suggests how I might re-educate myself (hint: I _have_ actually installed and played with it). * I go ahead and implement the solution proposed above. I'm confident that my employer would be keen on release under a DFSG-compatible license (probably MIT, since I'd be using Ruby). I have a handle on every aspect of the solution except for one: the feed consumer for DSAs. I've reverse-engineered debsecan, and mostly have my head around the security tracker's suite feeds. I'm talking about the feeds available at URLs like: http://secure-testing.debian.net/debian-secure-testing/project/debsecan/release/1/etch I just need some clarity: * I'd have to consume feeds for multiple suites, (because we use backports and volatile), and would have to do some version munging to deal with things like ~bpo in versions. I could get package suites with something like apt-show-versions. Am I on the right track here? * I get what the etch, lenny, sid suites are for. What's in the GENERIC feed? * There are fields in the feeds that debsecan calls unstable_version and other_versions. As I read the code, a package is vulnerable if: pkg_version unstable_version pkg_version == any of other_versions This seems unlikely to me, and I suspect I've misread the code. Can someone clarify? Again, to be clear, I realize that we need to work towards update policy and mechanisms that guarantee that security updates are applied automatically and in a timely manner. We're working on that, but it's going to take us time. And even once we're done, we'll still want a vulnerability tracker for some time thereafter, until we're confident that things are running smoothly. So any suggestions on how to implement scalable Debian vulnerability tracking in the interrim would be greatly appreciated. Thanks, Sheldon. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) iD4DBQFJY8BUpGJX8XSgas0RAlrWAJ9/PubX5OTma2DwbPy+NfUcR8k7xQCVHR3w Dsk1udom6T+JZzDX0Y86LA== =tdtx -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Scalable Debian vulnerability tracking
Hi folks, I work for an hosting provider, and am looking at how to improve visibility into vulnerability exposure. We have over 800 Debian hosts that we manage fore customers, and will have over 1,000 by the end of this quarter. A major problem we face is that our change distribution mechanism is poor. We're working on that problem, but in the meantime, I'm looking at ways to assert that we are / are not vulnerable to specific issues disclosed by the Debian project. I realize that this isn't the whole game, but it's an huge part of it. First prize is a web application that we can draw reports from (or will push reports to us or whatever), that knows what security issues have been identified and addressed by the Debian project, what versions of packages are installed on all servers, and therefore which packages on which servers should have been upgraded but have not yet been. Yup, basically the output of debsecan --only-fixed --suite etch. But I'd prefer not to use email as the transport mechanism (unreliable), and I'd have to write an aggregator for all those mails, because working through mail from over a thousand servers is error prone. [...] This is definitely not a complete solution to your problem, but it might help you along the way: - Run apt-get update + apt-show-versions on each host (daily, hourly, whatever you like) - If you don't like email for aggregation, a central syslog may be an option. Pipe the output of apt-show-versions through logger and filter and aggregate the logs on your server. We don't have hundreds of servers, but this scheme works fairly well around here, using a very simply daily cron job and logwatch as the aggregator. HTH, Michael pgpb2jmR3RIjQ.pgp Description: PGP signature
Re: Scalable Debian vulnerability tracking
I have a package that we have been working on for a while that might be a good starting point. It tracks information about several machines, storing them in a central repository. There is a client piece installed on each machine which runs on a cron job, and currently e-mails the results to one or more centralized servers. I wrote it with the idea in mind that other transport agencies could be used, however. The server piece is a php app w/MySQL backend that allows reporting, including ad-hoc reports. The other functions such as maintenance tracking would probably not be of interest to you, but those are just modules that can be disabled (in the new version we're working on). The UI is primitive and ugly. Again, the package is pretty primitive, but has been tested on a dozen Debian servers and two Fedora servers over the past year. It has one report that would be of interest to you that retrieves the current version of a package installed on all (or some) machines. This is gpl'd, and I would be happy to supply the .deb, the source tree or svn access if you would like to look at it. You can see some very, very outdated documentation at http://wiki.linuxservertech.com/index.php/Sysinfo. That documentation is is about one major version behind, and the links are to the older version. I'm making some big changes right now. Rodo (at this domain to email directly) Michael Tautschnig wrote: Hi folks, I work for an hosting provider, and am looking at how to improve visibility into vulnerability exposure. We have over 800 Debian hosts that we manage fore customers, and will have over 1,000 by the end of this quarter. A major problem we face is that our change distribution mechanism is poor. We're working on that problem, but in the meantime, I'm looking at ways to assert that we are / are not vulnerable to specific issues disclosed by the Debian project. I realize that this isn't the whole game, but it's an huge part of it. First prize is a web application that we can draw reports from (or will push reports to us or whatever), that knows what security issues have been identified and addressed by the Debian project, what versions of packages are installed on all servers, and therefore which packages on which servers should have been upgraded but have not yet been. Yup, basically the output of debsecan --only-fixed --suite etch. But I'd prefer not to use email as the transport mechanism (unreliable), and I'd have to write an aggregator for all those mails, because working through mail from over a thousand servers is error prone. [...] This is definitely not a complete solution to your problem, but it might help you along the way: - Run apt-get update + apt-show-versions on each host (daily, hourly, whatever you like) - If you don't like email for aggregation, a central syslog may be an option. Pipe the output of apt-show-versions through logger and filter and aggregate the logs on your server. We don't have hundreds of servers, but this scheme works fairly well around here, using a very simply daily cron job and logwatch as the aggregator. HTH, Michael -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Scalable Debian vulnerability tracking
In gmane.linux.debian.devel.security, you wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi folks, I work for an hosting provider, and am looking at how to improve visibility into vulnerability exposure. We have over 800 Debian hosts that we manage fore customers, and will have over 1,000 by the end of this quarter. A major problem we face is that our change distribution mechanism is poor. We're working on that problem, but in the meantime, I'm looking at ways to assert that we are / are not vulnerable to specific issues disclosed by the Debian project. I realize that this isn't the whole game, but it's an huge part of it. First prize is a web application that we can draw reports from (or will push reports to us or whatever), that knows what security issues have been identified and addressed by the Debian project, what versions of packages are installed on all servers, and therefore which packages on which servers should have been upgraded but have not yet been. Yup, basically the output of debsecan --only-fixed --suite etch. But I'd prefer not to use email as the transport mechanism (unreliable), and I'd have to write an aggregator for all those mails, because working through mail from over a thousand servers is error prone. So I imagine a solution involving: * a web service to which servers can basically post their dpkg status, * a feed consumer for DSAs, * a vulnerability resolver that matches these two data sources, * reports to provide a view into the persisted resolver results. I spent a lot of time hunting for such a thing, and haven't found one I'm happy with. I'm not keen on using Nessus for this, for a few reasons: * I don't think it's an hosted application with persisted results (i.e. you have to initiate a full network-wide test every time you want a report). * The Tenable sales people are useless. * I don't think their feeds are worth the money. OpenVAS would address the last two, but I can't recommend it in production on Debian Etch or Lenny yet, and there's still the matter that it's not an hosted application with persisted results. So I'm hoping for one of: * Someone points me to an existing app, I bow my head in shame, admit I have no Google Fu, and avoid reinventing the wheel. * Someone points out how wrong I am about Nessus and suggests how I might re-educate myself (hint: I _have_ actually installed and played with it). * I go ahead and implement the solution proposed above. I'm confident that my employer would be keen on release under a DFSG-compatible license (probably MIT, since I'd be using Ruby). I have a handle on every aspect of the solution except for one: the feed consumer for DSAs. I've reverse-engineered debsecan, and mostly have my head around the security tracker's suite feeds. I'm talking about the feeds available at URLs like: http://secure-testing.debian.net/debian-secure-testing/project/debsecan/release/1/etch I just need some clarity: * I'd have to consume feeds for multiple suites, (because we use backports and volatile), and would have to do some version munging to deal with things like ~bpo in versions. I could get package suites with something like apt-show-versions. Am I on the right track here? [..] So any suggestions on how to implement scalable Debian vulnerability tracking in the interrim would be greatly appreciated. Hi Sheldon, that sounds like an interesting project. Thanks for the intent to share and to provide your possible solution it to the public! For an environment the size of yours I'd recommend to implement a Nagios plugin, which checks the local state of installed applications against the data from the Debian Security Tracker, which is also used for debsecan. The Security Tracker URL is http://security-tracker.debian.net/tracker/ The whole data is available in SVN under svn://svn.debian.org/secure-testing/ (don't let you be fooled by the name, it also applies for stable and backports.org, the repository name is due to historic reasons). All the data is below data/, the code to parse the data is below lib/python/ and bin/. The existing scripts should provide an easy base to build upon. Beware that there might a some false positives, all data in debsecan and the Debian Security Tracker is initially flagged as vulnerable for stable-security unless it's properly validated or fixed, so it can happen that a package is marked as vulnerable for Etch, which is actually safe. We have an extensive HOWTO on how the Debian Security Tracker works: http://svn.debian.org/wsvn/secure-testing/doc/narrative_introduction?op=filerev=0sc=0 If you want to amend/correct data, please send mail to debian-security-trac...@lists.debian.org. (This is also the mailing list for followup questions on the Tracker or the code) There are also people available on the OFTC IRC network on #debian-security who should be able to help you out. BTW
Re: Scalable Debian vulnerability tracking
are for. What's in the GENERIC feed? * There are fields in the feeds that debsecan calls unstable_version and other_versions. As I read the code, a package is vulnerable if: pkg_version unstable_version pkg_version == any of other_versions This seems unlikely to me, and I suspect I've misread the code. Can someone clarify? Again, to be clear, I realize that we need to work towards update policy and mechanisms that guarantee that security updates are applied automatically and in a timely manner. We're working on that, but it's going to take us time. And even once we're done, we'll still want a vulnerability tracker for some time thereafter, until we're confident that things are running smoothly. So any suggestions on how to implement scalable Debian vulnerability tracking in the interrim would be greatly appreciated. Thanks, Sheldon. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) iD4DBQFJY8BUpGJX8XSgas0RAlrWAJ9/PubX5OTma2DwbPy+NfUcR8k7xQCVHR3w Dsk1udom6T+JZzDX0Y86LA== =tdtx -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Best Regards, -- Jonás Andradas Skype: jontux LinkedIn: http://www.linkedin.com/in/andradas GPG Fingerprint: 5A90 3319 48BC E0DC 17D9 130B B5E2 9AFD 7649 30D5 Keyservers: wwwkeys.eu.pgp.net | pgp.rediris.es
Re: Scalable Debian vulnerability tracking
Is there anything wrong with using cfengine for this? [1] I'd just have a very simple layout for cfengine files and a cf.packages.$distro [2] file for each distro we support. Then have cfengine maintain a list of known packages that needs to be on each. Reporting can be easily done from a module (cfengine module). And this can be written in whatever language you want. I've been successfully using cfengine to manage large datacenters for a long time. The learning curve is a bit steep at first, but you will be up and running in a few hours. Hope this helps. Regards, Notes: 1. http://www.cfengine.org/docs/cfengine-Reference.html 2. http://www.cfengine.org/docs/cfengine-Reference.html#packages -- )(- Luis Mondesi Maestro Debiano - START ENCRYPTED BLOCK (Triple-ROT13) -- Gur Hohagh [Yvahk] qvfgevohgvba oevatf gur fcvevg bs Hohagh gb gur fbsgjner jbeyq. - END ENCRYPTED BLOCK (Triple-ROT13) -- -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Scalable Debian vulnerability tracking [REDUX]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tuesday 06 January 2009 23:28:29 Erik Harrison wrote: actually, if you do find something that does this, dont publish anything. start a company. No thanks. Just got out of that game and am thoroughly enjoying being an employee in an organisation that recognizes the value of open source and encourages contribution. :-) On Wednesday 07 January 2009 06:39:53 Luis Mondesi wrote: Is there anything wrong with using cfengine for this? Nothing wrong with that. In fact, we'll be using Puppet instead. But yes, it would certainly be better to just keep up to date with updates in a near-fully automated fashion. But we won't be done with that this quarter, and for some time after we're done, we'll still want to monitor that we're getting it right. On Wednesday 07 January 2009 02:44:46 Jonas Andradas wrote: If you can modify the mail system of each host, you could create an exim4 router and transport that will accept mail for a local user (lets call it SecMonitor) which would pipe the output of debsecan to a script (be it shell, perl, python or anything you choose). Interesting. I hadn't considered letting debsecan do the work but obviating SMTP as the transport protocol. I'll have a think about whether I like the idea of leaving the work of the vulnerability resolver up to the client hosts. It opens the door for decentralized local policy configuration, which is good and bad. :-) On Wednesday 07 January 2009 00:24:09 R. W. Rodolico wrote: I have a package that we have been working on for a while that might be a good starting point. This is gpl'd, and I would be happy to supply the .deb, the source tree or svn access if you would like to look at it. Suppressing my knee-jerk reaction to PHP, it sounds like you're quite far down the track with this one. :-) I'd love to see what you've got, and from the sound of it, at least one other person on this list would like to see it too. SVN access would be first prize, but a source tarball might be easier for you to arrange if you elect to disclose a URL here. On Wednesday 07 January 2009 01:45:36 Moritz Muehlenhoff wrote: We have an extensive HOWTO on how the Debian Security Tracker works: http://svn.debian.org/wsvn/secure-testing/doc/narrative_introduction? op=filerev=0sc=0 This is gold, thank you. Is there also an explanation of the file format of the debsecan feeds, though? By debsecan feeds, I mean the libz-compressed files such as http://secure-testing.debian.net/debian-secure-testing/project/debsecan/release/1/etch In particular, I'm still unclear on how to interpret the unstable_version and other_versions fields. Thanks, Sheldon. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) iD8DBQFJZFLGpGJX8XSgas0RAjhIAKCTqiJZUvzcm2IJUxHNJft8Mn87WwCdHspz i2Kn6xSM2EHJSjHYGZ+HI5I= =FMK9 -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
