Thanks for the reply Luca. Here is the full command line I'm using:
ntopng -i tcp://127.0.0.1:5556 -i tcp://127.0.0.1:2001 -i tcp://127.0.0.1:2013 -i tcp://127.0.0.1:2018 -m [LOCALADDRESSES] -n 1 -X 200000 -p /etc/ntopng/protos.txt We have multiple devices sending flow data to different ports and I have nprobe running as: nprobe --zmq tcp://*:5556 -i none -n none --collector-port [INCOMINGPORT] I have one nprobe running for each port on which I'd like to collect data. The majority of the flow data is coming through the 5556 port and I've tried just listening on one port with one nprobe collector and have DNS cache disappear. Even if I set the default -n 0 option the DNS resolution still halts after some time. I've re-run my tests with the current SVN version and I'm still having the resolution issues (here is the version reported via the web interface - Generated by ntopng v.1.1.1 (r7147)). How often are records in the dns.toresolve list popped to resolve? Is there a way I can increase the number of records popped or increase the frequency with which they are popped? Thanks again for the assistance. Cheers, Andrew. -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Luca Deri Sent: Monday, January 6, 2014 11:56 To: [email protected] Cc: [email protected]; [email protected] Subject: Re: [Ntop-dev] DNS name resolution stops working Andrew before tuning the system, i would like to understand how you have configured ntong. Can you please send me the complete command line you have used? By default we resolve only the local addresses as if you have a heavy loaded network the behaviour you describe is normal. Please re-run your tests using the latest ntopng code in SVN Regards Luca On 03 Jan 2014, at 16:50, Andrew Rechenberg Lists <[email protected]> wrote: > Good day, > > I'm using ntopng 1.1 and redis 2.8.3 on CentOS 6.5 x86_64 on a system with 2 > dual core Xeon 5130 2.00GHz with 8GB of RAM. When ntopng is first started > name resolution works fine. After a short period of time DNS name resolution > stops working. > > I understand that the name records are store in redis and looking at the code > they are cached for 300 seconds. When I look at the redis keys after about 5 > minutes of ntopng uptime most of the dns cache records have been replaced > IP.json records. The dns.toresolve list in redis is also almost always at > the 500 record limit. > > If I trim the dns.toresolve list a few times by using the redis-cli command: > > LTRIM dns.toresolve 0 0 > > and then refresh the Active Flows list a few times then the host IPs are then > resolved to names. > > The web interface shows 2300-2600 hosts and 25000-140000 flows. > > Currently the number of keys in the redis database range from about 512-2000. > It appears that the redis database isn't growing to accommodate all of the > hosts that our instance needs to store. The redis-server process is running > with the default options and is consuming about 50MB of system RAM. I tried > changing the maxmemorypolicy to noeviction but that didn't seem to make a > difference. > > It also appears that the name resolution queue isn't being processes fast > enough and the most frequent hosts with flows are having their dns.cache.* > records expire in redis and never get placed back on the queue because it is > already 500+ > > What can be done to make sure that DNS name resolution always operates > properly? Do I need to change the MAX_NUM_QUEUED_ADDRS define? Is there a > way that I can increase the number of keys that the redis database will store > in memory? Is there a way to process the DNS host resolution queue faster? > > Thank you for your asssitance. > > Cheers, > Andrew. > Confidentiality Notice: This e-mail message including attachments, if any, is > intended only for the person or entity to which it is addressed and may > contain confidential and/or privileged material. Any unauthorized review, > use, disclosure or distribution is prohibited. If you are not the intended > recipient, please contact the sender by reply e-mail and destroy all copies > of the original message. If you are the intended recipient, but do not wish > to receive communications through this medium, please so advise the sender > immediately. > _______________________________________________ > Ntop-dev mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-dev _______________________________________________ Ntop-dev mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-dev Confidentiality Notice: This e-mail message including attachments, if any, is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. If you are the intended recipient, but do not wish to receive communications through this medium, please so advise the sender immediately. _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
