On Fri, Jul 18, 2003 at 01:35:55PM +0200, Renaud Deraison wrote:
> On Thu, Jul 17, 2003 at 04:10:17PM -0700, Fyodor wrote:
> By the way Nessus is designed, that would do something like :
>
> while(next_host_to_scan)
> {
> hosts[i++] = some_host;
> if(i >= MAX_HOSTS_PER_SPECIAL_PLUGIN)
> {
> scan_host_in_background_with_special_plugins(hosts);
> for(i=0;i<MAX_HOSTS_PER_SPECIAL_PLUGIN;i++)
> scan_host_in_background(hosts[i]);
> }
> }
True, that version is 4 lines longer. But it could help you produce
better results AND save the 18,980 lines that you imported into the
nmap_osfingerprint plugin. And yes, I do understand that your actual
implementation would take more than 4 extra lines, but perhaps not
more than 400.
> This is starting to look non-pretty. But wait, we usually want to ping
> the hosts before scanning them. I prefer Nessus's ping to nmap's ping,
> which means I have to use another plugin and create a category
> ACT_PING.
If you tell me why you prefer the Nessus ping, I may be able to fix
whatever Nmap limitation you are concerned about. The Nmap ping
facility works in parallel, allows you to specify an arbitrary
combination of probes (including multiple TCP syns, TCP acks, UDP
probes, ICMP echo request, timestamp request, netmask request, etc).
It dynamically calculates latencies and packet loss rates (which are
then used to speed up the port/OS scan) and it handles
retransmissions. Ping scanning tens of thousands of hosts goes pretty
quickly. I am not saying it is perfect, but if you tell me your
complaint I will investigate.
> I can not just sequentially ping the hosts, that would be a waste of
> time, so I need to launch the plugin in background, read its input,
> and see if I need to continue the scan.
See above. If you tell me your concern with the Nmap ping scan, I may
be able to resolve it so that you can avoid this hassle.
> Which starts to be a bit more complex than my initial three lines of
> code. And this code is suboptimal :
> - We block the scan with a cluster of 64 hosts. Meaning that
> if one host is very slow to ping and portscan and to scan,
> the whole scan will wait for that guy to complete before
> going on.
Nmap will give you the results of each host as they become available.
You could use --host_timeout to specify the maximum amount of time a
host can take (and deal with the laggards later). If your initial
scan is TCP only and uses (say) the ~1600 ports Nmap scans by default,
it should not take too long even against a heavily firewalled host.
> So the code would be bloated, bug-prone, suboptimal and more importantly
> it would totally break the architecture around which nessusd was
> designed.
"totally break the architecture"? I would have used the term
"enhance" or "improve" instead :). Is Nmap the only plugin that would
benefit from being able to handle more than one host at a time? One
instance of Nmap can easily scan a class B. But Nessus handles this
by starting 65,536 copies of Nmap, often having many dozens of them
running at once.
> > and to interpret
> > the Nmap XML output instead of the human-readable or "grepable"
> > formats. The XML output is designed for this and almost never changes
> > in incompatible ways.
>
> That would require the linking of libxml. Or libxml2. Nice support
> issues in perspective for very little benefits compared to the use of
> the human-readable format.
The XML format is best, but Nmap also offers a "grepable" format (-oG
or -oM) which almost never changes. You can parse the "human
readable" format, but it will change on occasion. That is why I went
to the trouble of creating the other 2 formats.
> The real point with the use of the human-readable output is that people
> can do a scan and upload it to nessusd at the same time they can read
> it. And the human beings I know actually prefer to read a text file than
> an XML file. I don't know why.
Nmap will produce both. If you use '-oG /path/for/nessus-to-read/log.nmap',
normal output still goes to stdout (-oG is the grepable output
discussed above, -oX would give you XML). You can also add -oN to
save the "human readable" format to a different file while you are
sending -oG output to Nessus.
> And to get back to your original statement : yes, the nessus download
> page used to recommand nmap when installing Nessus. I still recommand
> nmap to people who want to "pen-test" a single host. To people who want
> to scan a whole network, I do not recommand Nmap, because it uses too much
> memory (more than 25MB per host), which in turns makes Linux crash
First of all, I am not sure where you get the 25M number. I just
tried on OS scan on my Linux box and it showed under 10M in top:
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1540 root 10 0 77332 28M 10740 R 2.1 3.8 54:14 X
29706 fyodor 9 0 70224 68M 21092 S 0.0 9.0 0:00 mozilla-bin
6996 fyodor 9 0 11212 10M 6956 R 0.1 1.4 0:06 acroread
17404 fyodor 9 0 9888 9180 2388 R 0.0 1.1 12:38 xmms
7108 root 11 0 9844 9844 912 R 9.0 1.2 0:00 nmap
But you statement above mentions the memory use as "per host". This
is again because Nessus is starting dozens of Nmap processes in
parallel, which Nmap wasn't designed for. So this is yet another
advantage of passing multiple hosts to Nmap at once. Run one copy of
Nmap against 64 machines and it will use up 10M of virtual memory.
Start up 64 copies at once and the requirements balloon to 640M. Or
25M/1600M using your number.
> And the 25MB of memory are pure bloat - nmap-os-fingerprints takes
> 300kb on disk - how come it becomes a 25MB monster once loaded ?
The old version which comes with Nessus may be 300K, but the current
nmap-os-fingerprints is more than 450K. It detects hundreds of OS
types that the old version in Nessus does not know about. And then
users think Nmap is the problem when OS detection fails using the
Nessus-internal version :(.
> The fact is that I have numerous reports of people who went to
> nessus.org, saw that Nmap was "mandatory", cursed about the huge list of
> dependencies, installed it anyway,
You don't need to list it as mandatory. You could put "optional but
heavily recommended") as you presently do with OpenSSL on the download
page. Then if users ignore the advice, they may understand why the
results suck. Right now they are often confused about how Nessus is
obtaining the results (for example, see the message I replied to
on this list yesterday).
I never said it would be trivial to change Nessus so it calls
designated plugins with more than one host at a time. But I find it
hard to believe that maintaining an entire copy of the Nmap source
tree within Nessus will be easier. I believe speed and accuracy will
suffer (and has suffered) in the latter case as well. But it is your
scanner and of course you are free to do as you wish.
Cheers,
Fyodor
http://www.insecure.org/
