Send netdisco-users mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.sourceforge.net/lists/listinfo/netdisco-users
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of netdisco-users digest..."
Today's Topics:
1. Re: better control of cpu-performance impact (Oliver Gorwits)
2. Re: better control of cpu-performance impact (Andy Ruhl)
3. Re: better control of cpu-performance impact ([email protected])
4. Re: better control of cpu-performance impact ([email protected])
--- Begin Message ---
Hi Piotr,
This is an interesting scenario and set of questions! You have not said
how often you're triggering macwalk and arpwalk. The default is once per
hour - how long is it taking to cover your 4k devices, and are you
scheduling macwalk and arpwalk more than once per hour?
Unfortunately there is no current way to reserve workers for an action
such as macsuck or arpnip. It would also be quite difficult to implement
this, I think. There is also no point running two backends on the same
server, as that would be no different to running one backend and
configuring more workers (unless I've misunderstod?).
The best approach might be:
1. check the UDP buffer size on your network interface on Linux. I have
seen UDP buffer overruns before and it's interesting that you had better
experience on BSD.
2. start a second *server* and run a second backend there. You can
disable the scheduler using the config:
schedule: null
In general the database likes more memory, and the pollers like more
cores. 4 cores (or 8 with hyperthreading?) is not very much for 4k
devices. If you can increase the number of cores in your VM then do so,
or else run more VMs. You can also move the DB off to its own server as
well to free more cores.
If I was to have a wishlist feature to help here, it would be to have a
listener in Netdisco for SNMP traps and then configure your devices to
send SNMP traps when ports go up/down. That way you would get even
up-to-date info. It would not be too difficult to do this (right now the
devs have a lot to work on... but we can give advice, and patches are
welcome!).
regards,
oliver.
On 2018-02-27 10:41, Piotr Blaszczyk wrote:
Hi @ All
im using Netdisco in a enviroment of ~4k devices and ~116k clients
the ability to have accurate timestamps of nodes, when they move or go
offline,
is quite handy in network support
thats why im trying to cycle the network with macsuck & arpnip as often
as
possible
in a vm with 32gb ram (db+netdisco), 4core xeon and 17 workers, the cpu
will
spike to 100%, but overall remains between ~ 40% - 80% while
discoverall and
macwalk
arpwalk will constantly push the CPU to 100% and while i didnt had
problems on
freebsd, on SLES this results in packet drops on the network interface
the idea run different pollers (1 for macsuck&discoverall and 1 for
arpwalk) on
the same server with different numbers of workers could solve my
problem, but im
not sure if i can use the same machine for 2 pollers
@Oliver
is it possible to build-in optional parameters like discover_workers /
arp_walkers etc ?
greetings
piotr
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Netdisco mailing list
[email protected]
https://sourceforge.net/p/netdisco/mailman/netdisco-users/
--- End Message ---
--- Begin Message ---
It's not so surprising to me that BSD works better than Linux, but I
haven't studied this for many years now.
Some general advice:
It's a really good idea to run the database server on another machine,
just on principle. Also look into disk layout, use a separate disk for
the log if your disk is busy.
Use multiple bonded links for your network interface if possible. I
use LACP, seems to work well.
Don't schedule a job to run more often than it needs to. Spread stuff
out. I don't need to have up to the minute port usage stats.
Andy
--- End Message ---
--- Begin Message ---
Hi Oliver,
schedule looks like:
discoverall:
when: '30 5,21 * * *'
macwalk:
when: '0 7-19/2 * * 1-5'
arpwalk:
when: '20 8-20/2 * * 1-5'
nbtwalk:
when: '35 8 * * 1-5'
expire:
when: '0 2 * * *'
CPU-Graph looks like:
https://imgur.com/a/PojMm
and the idea with 2 different pollers looks like
poller 1:
schedule:
discoverall:
when: '30 5,21 * * *'
macwalk:
when: '0 7-19/2 * * 1-5'
arpwalk:
when: null
nbtwalk:
when: '35 8 * * 1-5'
expire:
when: '0 2 * * *'
workers:
tasks: '17'
sleep_time: 1
poller 2:
schedule:
discoverall:
when: null
macwalk:
when: null
arpwalk:
when: '35 8 * * 1-5'
nbtwalk:
when: null
expire:
when: null
workers:
tasks: '3'
sleep_time: 1
im quite happy with the performance of the system. knowing solutions like
solawinds orion or nedi where search requests are quite disruptive in workflow
(slow), a search in netdisco is done realy fast
SNMP traps sounds interesting, but afaik we are not unsing it
greetings
piotr
> Oliver Gorwits <[email protected]> hat am 27. Februar 2018 um 19:03 geschrieben:
>
>
> Hi Piotr,
>
> This is an interesting scenario and set of questions! You have not said
> how often you're triggering macwalk and arpwalk. The default is once per
> hour - how long is it taking to cover your 4k devices, and are you
> scheduling macwalk and arpwalk more than once per hour?
>
> Unfortunately there is no current way to reserve workers for an action
> such as macsuck or arpnip. It would also be quite difficult to implement
> this, I think. There is also no point running two backends on the same
> server, as that would be no different to running one backend and
> configuring more workers (unless I've misunderstod?).
>
> The best approach might be:
>
> 1. check the UDP buffer size on your network interface on Linux. I have
> seen UDP buffer overruns before and it's interesting that you had better
> experience on BSD.
>
> 2. start a second *server* and run a second backend there. You can
> disable the scheduler using the config:
>
> schedule: null
>
> In general the database likes more memory, and the pollers like more
> cores. 4 cores (or 8 with hyperthreading?) is not very much for 4k
> devices. If you can increase the number of cores in your VM then do so,
> or else run more VMs. You can also move the DB off to its own server as
> well to free more cores.
>
> If I was to have a wishlist feature to help here, it would be to have a
> listener in Netdisco for SNMP traps and then configure your devices to
> send SNMP traps when ports go up/down. That way you would get even
> up-to-date info. It would not be too difficult to do this (right now the
> devs have a lot to work on... but we can give advice, and patches are
> welcome!).
>
> regards,
> oliver.
>
> On 2018-02-27 10:41, Piotr Blaszczyk wrote:
> > Hi @ All
> >
> > im using Netdisco in a enviroment of ~4k devices and ~116k clients
> > the ability to have accurate timestamps of nodes, when they move or go
> > offline,
> > is quite handy in network support
> > thats why im trying to cycle the network with macsuck & arpnip as often
> > as
> > possible
> > in a vm with 32gb ram (db+netdisco), 4core xeon and 17 workers, the cpu
> > will
> > spike to 100%, but overall remains between ~ 40% - 80% while
> > discoverall and
> > macwalk
> > arpwalk will constantly push the CPU to 100% and while i didnt had
> > problems on
> > freebsd, on SLES this results in packet drops on the network interface
> > the idea run different pollers (1 for macsuck&discoverall and 1 for
> > arpwalk) on
> > the same server with different numbers of workers could solve my
> > problem, but im
> > not sure if i can use the same machine for 2 pollers
> >
> > @Oliver
> > is it possible to build-in optional parameters like discover_workers /
> > arp_walkers etc ?
> >
> > greetings
> > piotr
> >
> > ------------------------------------------------------------------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Netdisco mailing list
> > [email protected]
> > https://sourceforge.net/p/netdisco/mailman/netdisco-users/
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Netdisco mailing list
> [email protected]
> https://sourceforge.net/p/netdisco/mailman/netdisco-users/
>
--- End Message ---
--- Begin Message ---
Hi Andy,
im stuck with this server
since the db is only ~2GB its possible to optimize postgre to keep the most
stuff in memory
thanks for advices
greetings
Piotr
> Andy Ruhl <[email protected]> hat am 27. Februar 2018 um 19:42 geschrieben:
>
>
> It's not so surprising to me that BSD works better than Linux, but I
> haven't studied this for many years now.
>
> Some general advice:
>
> It's a really good idea to run the database server on another machine,
> just on principle. Also look into disk layout, use a separate disk for
> the log if your disk is busy.
>
> Use multiple bonded links for your network interface if possible. I
> use LACP, seems to work well.
>
> Don't schedule a job to run more often than it needs to. Spread stuff
> out. I don't need to have up to the minute port usage stats.
>
> Andy
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Netdisco mailing list
> [email protected]
> https://sourceforge.net/p/netdisco/mailman/netdisco-users/
>
--- End Message ---
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Netdisco mailing list - Digest Mode
[email protected]
https://lists.sourceforge.net/lists/listinfo/netdisco-users
