That doesn't sound right at all... Where did you get this info? -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Istvan Köpe Sent: Wednesday, April 28, 2010 9:34 AM To: [email protected] Subject: Re: [Ntop] how to monitor http and https only
I just learned that squid doesn't handle https. So is not good for what I need. On 28.04.2010 15:51, Gary Gatten wrote: > I'm sure iptables can log most everything as well. Someone has probably > written an app to format the logs and summarize the data. > > ----- Original Message ----- > From: [email protected]<[email protected]> > To: [email protected]<[email protected]> > Sent: Wed Apr 28 07:37:52 2010 > Subject: Re: [Ntop] how to monitor http and https only > > Once I had contact with squid and than I realized that squid is a whole > chapter in Linux... If is possible I prefer to not install any proxy. > > I. > > On 28.04.2010 13:55, Steve Clark wrote: > >> Or maybe something like squid proxy. I am pretty sure it keeps an >> access.log that could provide >> the info you are looking for. >> >> On 04/27/2010 05:08 PM, Gary Gatten wrote: >> >>> Ah, I see.... You just want to see if the users are "surfing" or >>> actually working? Not sure if nTop will give you this. The Domain >>> report will have some of this info, and rrd may actually store this >>> as well. I'm just not sure it will provide exactly what you seek. >>> What about "IP -> Summary -> Internet Domain"?, then drill down >>> from there? If this report will work for you, maybe run a script >>> with several "wget" on the appropriate URL's and save those each >>> night? Perhaps you could enable sticky hosts and then run a cron job >>> that restarts nTop at midnight (or whenever) each night? >>> >>> Are you wanting something like "WebSense" - that records every url >>> visited, the time of day, the time spent at each site, etc.? You may >>> want to check out "OpenDNS". They offer similar service for tracking >>> this type of info and it's not "too" expensive for small number of >>> users. I'm sure there is Open Source stuff that will do this as well >>> - I just don't know of any. >>> >>> nTop may be able to get what you want - it for sure will capture the >>> data, I just don't know of a predefined "report" that will show >>> exactly what you want. NTop is good at lots of things, but isn't a >>> perfect fit for everything. >>> >>> Maybe someone else will have other ideas. In the mean time I >>> recommend you play with nTop's options a little and see if you can >>> get what you need without being too convoluted. >>> >>> -----Original Message----- >>> From: [email protected] >>> [mailto:[email protected]] On Behalf Of Istvan Köpe >>> Sent: Tuesday, April 27, 2010 3:47 PM >>> To: [email protected] >>> Subject: Re: [Ntop] how to monitor http and https only >>> >>> The restrictions are done with iptables. There are only 4 hosts with >>> internet access(http and https only), 1 with full access(the manager) >>> and 1 test machine with full access. The rest are limited to antivirus >>> updates. >>> >>> I want to save which sites were visited by the users each day. I need 2 >>> type of reports: by local IP and by remote hosts. >>> >>> Istvan >>> >>> On 27.04.2010 23:35, Gary Gatten wrote: >>> >>>> With Sticky hosts, idle hosts are never purged from memory. >>>> Therefore, every new host will take more and more until it runs >>>> out. Depending on the number of hosts, I can't tell you if 256MB >>>> will be enough or not. My guess is not. >>>> >>>> Maybe Wireshark is all you need? A capture filter will limit your >>>> traffic to http (or whatever) and you can tell it to create a new >>>> file every hour / 100MB / whatever. Then, some of the summary >>>> reports may give the info you need. If you don't capture DNS >>>> traffic you may have a hard time reconciling host ip's to urls, so >>>> keep that in mind. >>>> >>>> If you're trying to solve a specific problem or answer a specific >>>> question, perhaps post that? >>>> >>>> G >>>> >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: [email protected] >>>> [mailto:[email protected]] On Behalf Of Istvan Köpe >>>> Sent: Tuesday, April 27, 2010 3:29 PM >>>> To: [email protected] >>>> Subject: Re: [Ntop] how to monitor http and https only >>>> >>>> After all I don't even need graphs, but everywhere I looked, everybody >>>> is suggesting ntop, or maybe I'm not asking the right questions. >>>> >>>> What do you mean by "ntop memory usage continue to grow". The system >>>> running ntop is a piece of junk, with 256MB ram. Will it crash >>>> within 24h? >>>> >>>> Istvan >>>> >>>> On 27.04.2010 23:05, Gary Gatten wrote: >>>> >>>> >>>>> Sounds right. Beware: enabling sticky hosts will cause ntop memory >>>>> usage to continue to grow until: ntop is restarted, or ntop crashes >>>>> from a malloc error. >>>>> >>>>> There is probably a way to use "wget" and / or other tools to >>>>> "download" reports from ntop and save them somewhere. Then maybe >>>>> you could set idle purge for say... 70 minutes, and run this batch >>>>> report every hour? >>>>> >>>>> I think I understand what you're trying to do as I often need the >>>>> same thing. You may want to spend a few minutes looking at the >>>>> "rrd" settings. There may be some combination of "Data to Dump" >>>>> and "RRD Detail" that will do what you wish. I've played with >>>>> these settings some, but it's been a long time so can't offer much >>>>> guidance. There are several good docs on the web that give details >>>>> on what these settings do. If you can get RRD to store the data >>>>> you wish, you can then use the "Arbitrary Graph" option to fetch / >>>>> display that data. My initial thought is rrd will NOT store >>>>> "conversation" level info, but who knows - maybe somewhere in there >>>>> you'll get what you need? You could start be enabling all rrd data >>>>> sets at the "high" level. >>>>> >>>>> G >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: [email protected] >>>>> [mailto:[email protected]] On Behalf Of Istvan Köpe >>>>> Sent: Tuesday, April 27, 2010 2:47 PM >>>>> To: [email protected] >>>>> Subject: Re: [Ntop] how to monitor http and https only >>>>> >>>>> This means, if I want to see what web pages were opened by one >>>>> specific >>>>> user(local IP), I need to enable "sticky hosts" or I need to increase >>>>> purge hosts to 12 hours, right? >>>>> >>>>> I'll try with sticky hosts. That seems to be the closest to what I >>>>> need. >>>>> >>>>> Istvan >>>>> >>>>> On 27.04.2010 18:27, Gary Gatten wrote: >>>>> >>>>> >>>>> >>>>>> You may be speaking of two different issues: >>>>>> 1.) How nTop determines which hosts are local and which are remote >>>>>> 2.) Idle host purge timers >>>>>> >>>>>> First, please make sure you specify "-m all your local network >>>>>> ranges" on the command line. Or add via the GUI. This is the >>>>>> only way ntop knows local from remote. Anything not defined as >>>>>> local is considered remote. >>>>>> >>>>>> Next, the default idle host purge is 5 minutes. You have two >>>>>> options that I know of: >>>>>> 1.) Enable "sticky hosts" - which as implies hosts will never >>>>>> go away until you restart nTop. Only recommended in unique >>>>>> environments. >>>>>> 2.) Change the idle purge time in "globals-defines.h" and >>>>>> recompile nTop. >>>>>> >>>>>> Not sure which settings over ride which. If you make a change to >>>>>> the startup options, you must restart ntop and most/all recorded >>>>>> traffic will be lost. If done by the GUI, some settings are >>>>>> dynamic, I can't say for sure which ones. I think the GUI >>>>>> settings are saved in the prefsCache.db file. >>>>>> >>>>>> >>>>>> >>>>>> -----Original Message---- >>>>>> From: [email protected] >>>>>> [mailto:[email protected]] On Behalf Of Istvan Köpe >>>>>> Sent: Tuesday, April 27, 2010 10:06 AM >>>>>> To: [email protected] >>>>>> Subject: Re: [Ntop] how to monitor http and https only >>>>>> >>>>>> Even if I choose All protocols --> Traffic . I choose Hosts: >>>>>> All , I >>>>>> can't see all the remote hosts. But for a while I could see some >>>>>> remote >>>>>> hosts which than disappeared. What is the effective time range for >>>>>> All >>>>>> protocols --> Traffic ? >>>>>> Where are the parameters saved if I use the web interface for >>>>>> changing >>>>>> the configuration(Admin-->Configure-->Startup options)? >>>>>> I noticed that if I modify /etc/ntop.conf it overrides the web config >>>>>> settings. Is that right? >>>>>> If I modify the /etc/ntop.conf, how can I make the settings effective >>>>>> without losing the recorded traffic? >>>>>> >>>>>> On 27.04.2010 17:45, Gary Gatten wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> There's a startup arg to specify which network ranges are local, >>>>>>> it might be -b? Check the man and make sure you have this >>>>>>> configured correctly for your environment. >>>>>>> >>>>>>> ----- Original Message ----- >>>>>>> From: >>>>>>> [email protected]<[email protected]> >>>>>>> To: [email protected]<[email protected]> >>>>>>> Sent: Tue Apr 27 09:38:42 2010 >>>>>>> Subject: Re: [Ntop] how to monitor http and https only >>>>>>> >>>>>>> Ok, I got confused. Ntop is set on my Centos router. All the >>>>>>> internet >>>>>>> traffic goes through it. >>>>>>> I go on the web interface All protocols --> Traffic . I >>>>>>> choose Hosts: >>>>>>> Remote only and I see only some of the remote hosts. I don't >>>>>>> understand. >>>>>>> Where can I see all the remote hosts which were accessed today? >>>>>>> >>>>>>> Istvan >>>>>>> >>>>>>> On 26.04.2010 18:34, Gary Gatten wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> You can't disable "everything", but with packet and protocol >>>>>>>> filters, and by viewing specific reports - you can get pretty >>>>>>>> close to what you need. >>>>>>>> >>>>>>>> ----- Original Message ----- >>>>>>>> From: >>>>>>>> [email protected]<[email protected]> >>>>>>>> >>>>>>>> To: [email protected]<[email protected]> >>>>>>>> Sent: Mon Apr 26 09:31:35 2010 >>>>>>>> Subject: Re: [Ntop] how to monitor http and https only >>>>>>>> >>>>>>>> Thanks for the hints. But there is still too much information. >>>>>>>> All I want is: >>>>>>>> - 192.168.0.xxx, between 08:00-14:00, accessed the following >>>>>>>> sites: ... >>>>>>>> - www.facebook.com, between 08:00-14:00, was accessed by the >>>>>>>> following >>>>>>>> local IP-s: ... >>>>>>>> >>>>>>>> I don't need the: >>>>>>>> - Host Traffic Stats >>>>>>>> - Packet Statistics >>>>>>>> - Protocol Distribution >>>>>>>> - TCP/UDP Recently Used Ports >>>>>>>> - IP Service Stats: Client Role >>>>>>>> - TCP/UDP - Traffic on Other Ports >>>>>>>> >>>>>>>> How can I do all these? >>>>>>>> >>>>>>>> Istvan >>>>>>>> >>>>>>>> On 26.04.2010 17:12, Gary Gatten wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Good call. One can also restrict the displayed protocols with >>>>>>>>> -p, all remaining traffic will be displayed as "other" >>>>>>>>> >>>>>>>>> ----- Original Message ----- >>>>>>>>> From: >>>>>>>>> [email protected]<[email protected]> >>>>>>>>> >>>>>>>>> To: [email protected]<[email protected]>; >>>>>>>>> [email protected]<[email protected]> >>>>>>>>> Sent: Mon Apr 26 08:44:04 2010 >>>>>>>>> Subject: Re: [Ntop] how to monitor http and https only >>>>>>>>> >>>>>>>>> Have you taken a look at the manpages for ntop? On a unix >>>>>>>>> system, the "-B" switch followed by a pcap expression will give >>>>>>>>> you want you want. >>>>>>>>> >>>>>>>>> e.g >>>>>>>>> >>>>>>>>> ntop -d -w 8080 -B "port 80 or 443" >>>>>>>>> >>>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: [email protected] >>>>>>>>> [mailto:[email protected]] On Behalf Of Istvan >>>>>>>>> Köpe >>>>>>>>> Sent: Monday, April 26, 2010 9:40 AM >>>>>>>>> To: [email protected] >>>>>>>>> Subject: [Ntop] how to monitor http and https only >>>>>>>>> >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> I just installed ntop and it gives me much more information I >>>>>>>>> need. I >>>>>>>>> would like to see only the traffic on ports 80 and 443. >>>>>>>>> How can I do that? >>>>>>>>> >>>>>>>>> Istvan >>>>>>>>> _______________________________________________ >>>>>>>>> Ntop mailing list >>>>>>>>> [email protected] >>>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>>>> _______________________________________________ >>>>>>>>> Ntop mailing list >>>>>>>>> [email protected] >>>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>>>> _______________________________________________ >>>>>>>>> Ntop mailing list >>>>>>>>> [email protected] >>>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Ntop mailing list >>>>>>>> [email protected] >>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>>> _______________________________________________ >>>>>>>> Ntop mailing list >>>>>>>> [email protected] >>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> Ntop mailing list >>>>>>> [email protected] >>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>> _______________________________________________ >>>>>>> Ntop mailing list >>>>>>> [email protected] >>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> Ntop mailing list >>>>>> [email protected] >>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>> _______________________________________________ >>>>>> Ntop mailing list >>>>>> [email protected] >>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> Ntop mailing list >>>>> [email protected] >>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>> _______________________________________________ >>>>> Ntop mailing list >>>>> [email protected] >>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>> >>>>> >>>>> >>>>> >>>> _______________________________________________ >>>> Ntop mailing list >>>> [email protected] >>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>> _______________________________________________ >>>> Ntop mailing list >>>> [email protected] >>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>> >>>> >>>> >>> _______________________________________________ >>> Ntop mailing list >>> [email protected] >>> http://listgateway.unipi.it/mailman/listinfo/ntop >>> _______________________________________________ >>> Ntop mailing list >>> [email protected] >>> http://listgateway.unipi.it/mailman/listinfo/ntop >>> >>> >> >> > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop > _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
