Hi Andrew Today i done some tests to recreate your issues but I can not do it.
As anticipated by luca the behaviour you describe is normal because you can’t increase the frequency with which the records are popped. In order to increase the numbers of dns.toresolve list, you can increase the MAX_NUM_QUEUED_ADDRS define how you want. Please test this change and let me know if it works. Best regards, Filippo On 06 Jan 2014, at 22:15, Andrew Rechenberg Lists <[email protected]> wrote: > Thanks for the reply Luca. > > Here is the full command line I'm using: > > ntopng -i tcp://127.0.0.1:5556 -i tcp://127.0.0.1:2001 -i > tcp://127.0.0.1:2013 -i tcp://127.0.0.1:2018 -m [LOCALADDRESSES] -n 1 -X > 200000 -p /etc/ntopng/protos.txt > > We have multiple devices sending flow data to different ports and I have > nprobe running as: > > nprobe --zmq tcp://*:5556 -i none -n none --collector-port [INCOMINGPORT] > > I have one nprobe running for each port on which I'd like to collect data. > The majority of the flow data is coming through the 5556 port and I've tried > just listening on one port with one nprobe collector and have DNS cache > disappear. > > Even if I set the default -n 0 option the DNS resolution still halts after > some time. > > I've re-run my tests with the current SVN version and I'm still having the > resolution issues (here is the version reported via the web interface - > Generated by ntopng v.1.1.1 (r7147)). > > How often are records in the dns.toresolve list popped to resolve? Is there > a way I can increase the number of records popped or increase the frequency > with which they are popped? > > Thanks again for the assistance. > > Cheers, > Andrew. > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Luca Deri > Sent: Monday, January 6, 2014 11:56 > To: [email protected] > Cc: [email protected]; [email protected] > Subject: Re: [Ntop-dev] DNS name resolution stops working > > Andrew > before tuning the system, i would like to understand how you have configured > ntong. Can you please send me the complete command line you have used? By > default we resolve only the local addresses as if you have a heavy loaded > network the behaviour you describe is normal. > > Please re-run your tests using the latest ntopng code in SVN > > Regards Luca > > On 03 Jan 2014, at 16:50, Andrew Rechenberg Lists <[email protected]> > wrote: > >> Good day, >> >> I'm using ntopng 1.1 and redis 2.8.3 on CentOS 6.5 x86_64 on a system with 2 >> dual core Xeon 5130 2.00GHz with 8GB of RAM. When ntopng is first started >> name resolution works fine. After a short period of time DNS name >> resolution stops working. >> >> I understand that the name records are store in redis and looking at the >> code they are cached for 300 seconds. When I look at the redis keys after >> about 5 minutes of ntopng uptime most of the dns cache records have been >> replaced IP.json records. The dns.toresolve list in redis is also almost >> always at the 500 record limit. >> >> If I trim the dns.toresolve list a few times by using the redis-cli command: >> >> LTRIM dns.toresolve 0 0 >> >> and then refresh the Active Flows list a few times then the host IPs are >> then resolved to names. >> >> The web interface shows 2300-2600 hosts and 25000-140000 flows. >> >> Currently the number of keys in the redis database range from about >> 512-2000. It appears that the redis database isn't growing to accommodate >> all of the hosts that our instance needs to store. The redis-server process >> is running with the default options and is consuming about 50MB of system >> RAM. I tried changing the maxmemorypolicy to noeviction but that didn't >> seem to make a difference. >> >> It also appears that the name resolution queue isn't being processes fast >> enough and the most frequent hosts with flows are having their dns.cache.* >> records expire in redis and never get placed back on the queue because it is >> already 500+ >> >> What can be done to make sure that DNS name resolution always operates >> properly? Do I need to change the MAX_NUM_QUEUED_ADDRS define? Is there a >> way that I can increase the number of keys that the redis database will >> store in memory? Is there a way to process the DNS host resolution queue >> faster? >> >> Thank you for your asssitance. >> >> Cheers, >> Andrew. >> Confidentiality Notice: This e-mail message including attachments, if any, >> is intended only for the person or entity to which it is addressed and may >> contain confidential and/or privileged material. Any unauthorized review, >> use, disclosure or distribution is prohibited. If you are not the intended >> recipient, please contact the sender by reply e-mail and destroy all copies >> of the original message. If you are the intended recipient, but do not wish >> to receive communications through this medium, please so advise the sender >> immediately. >> _______________________________________________ >> Ntop-dev mailing list >> [email protected] >> http://listgateway.unipi.it/mailman/listinfo/ntop-dev > > _______________________________________________ > Ntop-dev mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-dev > Confidentiality Notice: This e-mail message including attachments, if any, is > intended only for the person or entity to which it is addressed and may > contain confidential and/or privileged material. Any unauthorized review, > use, disclosure or distribution is prohibited. If you are not the intended > recipient, please contact the sender by reply e-mail and destroy all copies > of the original message. If you are the intended recipient, but do not wish > to receive communications through this medium, please so advise the sender > immediately. > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
