On Tue, Nov 14, 2006 at 07:10:38PM +0100, Alef Veld wrote:
> If i wanted to discover a large amount of hosts, which method would
> be better. How does asynchronous work? Does it just send a bunch of
> udp packets in a threaded fashion?
>
> Another thing; as i start my foray in this snmp world :-), there seem
> to be _almost_ no librarys who deal directly with snmp. Almost all
> apps use a library like ucd or net-snmp. Why is this ? Although i'd
> rather code my own snmp client software i recognize the work and
> options that have been put into net-snmp so i will probably decide
> not to invent the wheel again.
>
> It's just that i find the net-snmp library pretty difficult to
> understand, and i wonder if there are some bare bones snmp client
> whose only hook is the systems headers/libraries.
The clear winner here is async: you can poll hundreds of hosts from
a modest machine with a single thread. This is because the great bulk
of the time spent by your program is *waiting for I/O*, and when it
works synchronously, this time is serialized.
You can ameliorate this somewhat by using multiple threads, but that
still serializes within a single thread. Threading introduces its own
complexity - especially since I don't think that NET-SNMP is fully
threadsafe - and this doesn't scale to hundreds of hosts.
I've seen single-threaded sync applications that have a very hard time
polling 100 hosts in 5 minutes, especially if one of them is down -
those retries and timeouts are all fully serialized, and your app is
doing nothing most of the time.
With async, I/O waits are 100% parallized, so you max out either on
your network bandwidth, or your CPU, but not on wall-clock time.
The core of your application will be an event loop. You'll have some
concept of a work-to-do list (hosts with lists of OIDs to request).
In each loop, you send as many requests as you can (up to some
reasonable limit), and the library keeps track of which ones are
waiting for a response.
Then you crank the I/O with a select() call, and hand the results to
the SNMP library. It runs through the list of file descriptors which
have fired, reads the responses, and matches them up with the requests
it's kept track of.
When it matches this response up, it fires your callback to process it,
and you'll take the response PDU, decode it, and store the data somewhere.
Then you tell the library that this request has been satisfied, and it
frees up one slot for another request you'll submit at the next loop.
Timeouts are handled as well, with the same callback mechanisms.
Make no mistake: async processing is much, much more complicated, and
even those who have been writing communications controllers for 20 years
*cough* me *cough* have some slow slogging.
But async is a slam-dunk win for performance.
Some time ago I wrote an async *DNS* resolver to crank through Apache
log files (doing IP to name resolutions), and it was able to do on the
order of 100 resolutions per second on an old slow machine.
fastzolver - Very fast Asynchronous DNS resolver for Webalizer
http://www.unixwiz.net/tools/fastzolver.html
In fact, most async apps require a way to pace themselves so they don't
blow out the infrastructure.
Steve (async kind of guy)
--
Stephen J Friedl | Security Consultant | UNIX Wizard | +1 714 544-6561
www.unixwiz.net | Tustin, Calif. USA | Microsoft MVP | [EMAIL PROTECTED]
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
