On Tue, Mar 04, 2003 at 07:32:51PM +0100, Javier Fernandez-Sanguino wrote:
> Well, I just found out that the next release of InfoSecurity Magazine (I
> don't have the paper version just yet) features a VA tools comparsion:
>
> http://www.infosecuritymag.com/2003/mar/cover.shtml
>
> It's unfortunate to see that there are wrong statements in the article
> such as: (when talking of plugin updates)
> "Nessus' vulnerability database is updated regularly, but you have to go
> to the Web site."
I saw the article (the author never told me it was on-line) and I saw
the errors. Apparently :
- Every plugin was enabled and the author was surprised to see stuff
crash
- Nmap was not installed on the host the journalist tested, therefore
tcp sequence prediction was not found
- The author was perfectly aware of nessus-update-plugins(8) [1]
- The author did not give SMB credentials when doing the Windows checks,
therefore Nessus obviously did not find dormant account. Had the
author given the proper credentials to Nessus, they would have been
found. The author was perfectly aware of that fact [1]
- the "OLD SSH" vulnerability does not mean anything. There's a serie
of OpenSSH 2.x.x which is not vulnerable to any flaw. We won't flag it
as being "old", that does not mean anything
- I told the author I would prefer the comparison to be done with the
GTK client, but he wanted to run it on Win32. I then suggested
NessusWX
<off topic>
> PS: There's also an interesting article on how to use the VA results to
> tune an IDS (Sourcefire) BTW.
And a link to Tenable Network Security. These guys rule :)
Everyone go to www.tenablesecurity.com !
</off topic>
I mailed the author and, suprisingly, he's "unavailable" until April
1st, and won't reply to e-mail nor telephone. Now that's a coincidence.
I'm obviously biased, so I'll let the members of the Nessus community
judge by themselves the professionalism of this test. The all e-mail
transcript between the author and myself is available on request.
I think this is a nice example of biased journalism, and that's very
sad.
[1] Below are two emails I sent to the author. Notice how I told him to
install Nmap, activate safe checks and give SMB credentials to Nessus.
Notice how I mentionned nessus-update-plugins twice.
Here we go :
------------------------------------------------------------------------------
On Tue, Oct 22, 2002 at 03:21:45PM -0700, Joel M Snyder wrote:
> Did you get a chance to read the invitation I added to the original
> message? It talks a great deal about the scenario I am going to test
> with, including the size of the network and the methodology. My
> thinking is that we are using this as a tool to help in tasks such as
> IDS and firewall tuning, and we will use it both inside and outside the
> network as appropriate to help with those.
Sorry, I read it too fast. Ok, I think Nessus should be included in the
review, it makes sense. I'd like you to test Nessus 1.2.6 (the latest
release) on RedHat 7.2 (or RedHat 8.0 with the package sharutils.rpm
installed) or FreeBSD 4.7. If you use RedHat, make sure that at
installation time you specified you wanted to include the developement
tools (you need a compiler and the GTK+ headers).
To install Nessus, either type:
lynx -source http://install.nessus.org | sh
or download it at
ftp://ftp.nessus.org/pub/nessus/nessus-1.2.6/nessus-installer/
and then execute:
./nessus-installer.sh
Please read the instructions carefully. Once Nessus is installed and you
have followed the instructions, you can update it using the utility
nessus-update-plugins(8) :
/usr/local/sbin/nessus-update-plugins
will fetch all the newest checks from www.nessus.org and install them at
the proper location. You may also want to install Nmap, as Nessus has
the ability to use it as an external plugin (make sure the nmap binary
is in your $PATH before you start nessusd).
Since you plan to play with IDSes, you may want to read
http://www.nessus.org/doc/nids.html, as Nessus contains some IDS evasion
features that may be fun to work with. Note that the screen shots are
slightly outdated, so things may not be exactly what they are on the web
site.
If you decide to scan Windows hosts and do so while being logged as a
domain administrator on ISS/Cybercop/whatever, be sure to fill the
"SMB login" and "SMB password" items in the 'Prefs' section of Nessus
(so that Nessus can, too, access the remote registry completely).
Do not hesitate to contact me or the Nessus Mailing list
(http://list.nessus.org) if you have any problem.
-- Renaud
------------------------------------------------------------------------------
On Mon, Dec 02, 2002 at 01:49:41PM -0700, Joel M Snyder wrote:
> Renaud:
>
> Thanks for all the notes. I am about to start scanning, and was going to use a
> Windows client---the Linux system nessus is installed on was built without a
> GUI. Which of the three do you want me to look at?
I suggest NessusWX[1], but keep in mind it's not the official client
Make also sure you updated your plugins with today's new sigs
(/usr/local/sbin/nessus-update-plugins).
[1] http://nessuswx.nessus.org
-- Renaud
------------------------------------------------------------------------------
On Tue, Dec 03, 2002 at 11:03:20AM -0700, Joel M Snyder wrote:
> >Finally, I'd have preferred you to use the official Nessus client (on
> >top of Linux), which offers more options than NessusWX in that regard,
> >as far as I recall.
>
> Well, I can start over from scratch and install it on a system with a GUI
> enabled, or I can keep going. Will it significantly change the results?
Yes and no. As NessusWX is not the official client, you're not really
judging Nessus.org per se. Some options are missing, but it has other
advantages, so that's your call. But if your goal is just to see "how
fast this goes", instead on focusing on the accuracy of the results,
then I'll have to suggest you to use Nessus.org's client. For instance,
I was told of a bug in NessusWX where "Disable all but dangerous
plugins" does not work as advertized. If you're judging Nessus.org on
that, then it's like judging MacOS X by complaining how Office X works
on it.
> I think that one of the important things about this is going to be issues such
> as differencing between runs. Are those features in the Linux client?
Yes, but I don't recommand using it at this time. It works, but there
are small issues to be ironed out.
-- Renaud
------------------------------------------------------------------------------
On Tue, Dec 03, 2002 at 11:34:53AM -0700, Joel M Snyder wrote:
> Well, you're probably making distinctions between pieces of the product that
> are not going to be that visible to other people who want to use it.
Except that it's officially distributed in a separate package.
> The one
> thing that I AM sure we want to do is make open source software be held to the
> same standards as commercial software---otherwise, it's not a fair comparison,
> and doesn't show them in the same light. So if I want to use a Windows client
> for everything else, then it's probably a reasonable thing to follow the
> Windows link on nessus.org which says 'Hey, I recommend you use this.' So far,
> I haven't seen any real reason to be unhappy with that client. It's not the
> most gorgeous interface out there, but it does get the job done fairly
> efficiently and quickly.
This is a very good client. My concern is that Nessus evolves at its own
speed and NessusWX follows the main Nessus tree as good as it can, but
not all the flashy features are integrated as soon as they are with the
main client. That's all.
> I'm not judging performance, except in a very broad way; on the other hand, I
> don't want it to take 7 days to finish the test since I have to get other
> scanners in and get them scanning away. So I am interested in tuning things
> with the kind of broad strokes you offered---a couple of tweaks here and there
> and the product is running about 10 times faster.
Then, fine. I had a problem in the past where speed was the only thing
taken in comparison for a review of security scanners, and Nessus did a
poor job, notably because it was not tested on the same platform as
the other scanners and it was not tuned at all.
> As I told you earlier, performance is not a major issue unless you're horribly
> awfully slow. But it appears that it's already completed a /24 in the time
> we've writing (about an hour), so things are going much faster now. I think
> that the other /24s will go even faster, because they are very sparsely
> populated. It shouldn't take long to analyze a /24 when there are only two
> hosts on it...
Right.
Thanks,
-- Renaud
