you need squid 2.5+ i guess http://www.comfsm.fm/computing/squid/FAQ-1.html#ss1.12
Istvan Köpe wrote: > I just learned that squid doesn't handle https. So is not good for what > I need. > > On 28.04.2010 15:51, Gary Gatten wrote: >> I'm sure iptables can log most everything as well. Someone has >> probably written an app to format the logs and summarize the data. >> >> ----- Original Message ----- >> From: >> [email protected]<[email protected]> >> To: [email protected]<[email protected]> >> Sent: Wed Apr 28 07:37:52 2010 >> Subject: Re: [Ntop] how to monitor http and https only >> >> Once I had contact with squid and than I realized that squid is a whole >> chapter in Linux... If is possible I prefer to not install any proxy. >> >> I. >> >> On 28.04.2010 13:55, Steve Clark wrote: >> >>> Or maybe something like squid proxy. I am pretty sure it keeps an >>> access.log that could provide >>> the info you are looking for. >>> >>> On 04/27/2010 05:08 PM, Gary Gatten wrote: >>> >>>> Ah, I see.... You just want to see if the users are "surfing" or >>>> actually working? Not sure if nTop will give you this. The Domain >>>> report will have some of this info, and rrd may actually store this >>>> as well. I'm just not sure it will provide exactly what you seek. >>>> What about "IP -> Summary -> Internet Domain"?, then drill down >>>> from there? If this report will work for you, maybe run a script >>>> with several "wget" on the appropriate URL's and save those each >>>> night? Perhaps you could enable sticky hosts and then run a cron job >>>> that restarts nTop at midnight (or whenever) each night? >>>> >>>> Are you wanting something like "WebSense" - that records every url >>>> visited, the time of day, the time spent at each site, etc.? You may >>>> want to check out "OpenDNS". They offer similar service for tracking >>>> this type of info and it's not "too" expensive for small number of >>>> users. I'm sure there is Open Source stuff that will do this as well >>>> - I just don't know of any. >>>> >>>> nTop may be able to get what you want - it for sure will capture the >>>> data, I just don't know of a predefined "report" that will show >>>> exactly what you want. NTop is good at lots of things, but isn't a >>>> perfect fit for everything. >>>> >>>> Maybe someone else will have other ideas. In the mean time I >>>> recommend you play with nTop's options a little and see if you can >>>> get what you need without being too convoluted. >>>> >>>> -----Original Message----- >>>> From: [email protected] >>>> [mailto:[email protected]] On Behalf Of Istvan Köpe >>>> Sent: Tuesday, April 27, 2010 3:47 PM >>>> To: [email protected] >>>> Subject: Re: [Ntop] how to monitor http and https only >>>> >>>> The restrictions are done with iptables. There are only 4 hosts with >>>> internet access(http and https only), 1 with full access(the manager) >>>> and 1 test machine with full access. The rest are limited to antivirus >>>> updates. >>>> >>>> I want to save which sites were visited by the users each day. I need 2 >>>> type of reports: by local IP and by remote hosts. >>>> >>>> Istvan >>>> >>>> On 27.04.2010 23:35, Gary Gatten wrote: >>>> >>>>> With Sticky hosts, idle hosts are never purged from memory. >>>>> Therefore, every new host will take more and more until it runs >>>>> out. Depending on the number of hosts, I can't tell you if 256MB >>>>> will be enough or not. My guess is not. >>>>> >>>>> Maybe Wireshark is all you need? A capture filter will limit your >>>>> traffic to http (or whatever) and you can tell it to create a new >>>>> file every hour / 100MB / whatever. Then, some of the summary >>>>> reports may give the info you need. If you don't capture DNS >>>>> traffic you may have a hard time reconciling host ip's to urls, so >>>>> keep that in mind. >>>>> >>>>> If you're trying to solve a specific problem or answer a specific >>>>> question, perhaps post that? >>>>> >>>>> G >>>>> >>>>> >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: [email protected] >>>>> [mailto:[email protected]] On Behalf Of Istvan Köpe >>>>> Sent: Tuesday, April 27, 2010 3:29 PM >>>>> To: [email protected] >>>>> Subject: Re: [Ntop] how to monitor http and https only >>>>> >>>>> After all I don't even need graphs, but everywhere I looked, everybody >>>>> is suggesting ntop, or maybe I'm not asking the right questions. >>>>> >>>>> What do you mean by "ntop memory usage continue to grow". The system >>>>> running ntop is a piece of junk, with 256MB ram. Will it crash >>>>> within 24h? >>>>> >>>>> Istvan >>>>> >>>>> On 27.04.2010 23:05, Gary Gatten wrote: >>>>> >>>>> >>>>>> Sounds right. Beware: enabling sticky hosts will cause ntop memory >>>>>> usage to continue to grow until: ntop is restarted, or ntop crashes >>>>>> from a malloc error. >>>>>> >>>>>> There is probably a way to use "wget" and / or other tools to >>>>>> "download" reports from ntop and save them somewhere. Then maybe >>>>>> you could set idle purge for say... 70 minutes, and run this batch >>>>>> report every hour? >>>>>> >>>>>> I think I understand what you're trying to do as I often need the >>>>>> same thing. You may want to spend a few minutes looking at the >>>>>> "rrd" settings. There may be some combination of "Data to Dump" >>>>>> and "RRD Detail" that will do what you wish. I've played with >>>>>> these settings some, but it's been a long time so can't offer much >>>>>> guidance. There are several good docs on the web that give details >>>>>> on what these settings do. If you can get RRD to store the data >>>>>> you wish, you can then use the "Arbitrary Graph" option to fetch / >>>>>> display that data. My initial thought is rrd will NOT store >>>>>> "conversation" level info, but who knows - maybe somewhere in there >>>>>> you'll get what you need? You could start be enabling all rrd data >>>>>> sets at the "high" level. >>>>>> >>>>>> G >>>>>> >>>>>> >>>>>> -----Original Message----- >>>>>> From: [email protected] >>>>>> [mailto:[email protected]] On Behalf Of Istvan Köpe >>>>>> Sent: Tuesday, April 27, 2010 2:47 PM >>>>>> To: [email protected] >>>>>> Subject: Re: [Ntop] how to monitor http and https only >>>>>> >>>>>> This means, if I want to see what web pages were opened by one >>>>>> specific >>>>>> user(local IP), I need to enable "sticky hosts" or I need to increase >>>>>> purge hosts to 12 hours, right? >>>>>> >>>>>> I'll try with sticky hosts. That seems to be the closest to what I >>>>>> need. >>>>>> >>>>>> Istvan >>>>>> >>>>>> On 27.04.2010 18:27, Gary Gatten wrote: >>>>>> >>>>>> >>>>>> >>>>>>> You may be speaking of two different issues: >>>>>>> 1.) How nTop determines which hosts are local and which are remote >>>>>>> 2.) Idle host purge timers >>>>>>> >>>>>>> First, please make sure you specify "-m all your local network >>>>>>> ranges" on the command line. Or add via the GUI. This is the >>>>>>> only way ntop knows local from remote. Anything not defined as >>>>>>> local is considered remote. >>>>>>> >>>>>>> Next, the default idle host purge is 5 minutes. You have two >>>>>>> options that I know of: >>>>>>> 1.) Enable "sticky hosts" - which as implies hosts will never >>>>>>> go away until you restart nTop. Only recommended in unique >>>>>>> environments. >>>>>>> 2.) Change the idle purge time in "globals-defines.h" and >>>>>>> recompile nTop. >>>>>>> >>>>>>> Not sure which settings over ride which. If you make a change to >>>>>>> the startup options, you must restart ntop and most/all recorded >>>>>>> traffic will be lost. If done by the GUI, some settings are >>>>>>> dynamic, I can't say for sure which ones. I think the GUI >>>>>>> settings are saved in the prefsCache.db file. >>>>>>> >>>>>>> >>>>>>> >>>>>>> -----Original Message---- >>>>>>> From: [email protected] >>>>>>> [mailto:[email protected]] On Behalf Of Istvan Köpe >>>>>>> Sent: Tuesday, April 27, 2010 10:06 AM >>>>>>> To: [email protected] >>>>>>> Subject: Re: [Ntop] how to monitor http and https only >>>>>>> >>>>>>> Even if I choose All protocols --> Traffic . I choose Hosts: >>>>>>> All , I >>>>>>> can't see all the remote hosts. But for a while I could see some >>>>>>> remote >>>>>>> hosts which than disappeared. What is the effective time range for >>>>>>> All >>>>>>> protocols --> Traffic ? >>>>>>> Where are the parameters saved if I use the web interface for >>>>>>> changing >>>>>>> the configuration(Admin-->Configure-->Startup options)? >>>>>>> I noticed that if I modify /etc/ntop.conf it overrides the web >>>>>>> config >>>>>>> settings. Is that right? >>>>>>> If I modify the /etc/ntop.conf, how can I make the settings >>>>>>> effective >>>>>>> without losing the recorded traffic? >>>>>>> >>>>>>> On 27.04.2010 17:45, Gary Gatten wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> There's a startup arg to specify which network ranges are local, >>>>>>>> it might be -b? Check the man and make sure you have this >>>>>>>> configured correctly for your environment. >>>>>>>> >>>>>>>> ----- Original Message ----- >>>>>>>> From: >>>>>>>> [email protected]<[email protected]> >>>>>>>> >>>>>>>> To: [email protected]<[email protected]> >>>>>>>> Sent: Tue Apr 27 09:38:42 2010 >>>>>>>> Subject: Re: [Ntop] how to monitor http and https only >>>>>>>> >>>>>>>> Ok, I got confused. Ntop is set on my Centos router. All the >>>>>>>> internet >>>>>>>> traffic goes through it. >>>>>>>> I go on the web interface All protocols --> Traffic . I >>>>>>>> choose Hosts: >>>>>>>> Remote only and I see only some of the remote hosts. I don't >>>>>>>> understand. >>>>>>>> Where can I see all the remote hosts which were accessed today? >>>>>>>> >>>>>>>> Istvan >>>>>>>> >>>>>>>> On 26.04.2010 18:34, Gary Gatten wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> You can't disable "everything", but with packet and protocol >>>>>>>>> filters, and by viewing specific reports - you can get pretty >>>>>>>>> close to what you need. >>>>>>>>> >>>>>>>>> ----- Original Message ----- >>>>>>>>> From: >>>>>>>>> [email protected]<[email protected]> >>>>>>>>> >>>>>>>>> >>>>>>>>> To: [email protected]<[email protected]> >>>>>>>>> Sent: Mon Apr 26 09:31:35 2010 >>>>>>>>> Subject: Re: [Ntop] how to monitor http and https only >>>>>>>>> >>>>>>>>> Thanks for the hints. But there is still too much information. >>>>>>>>> All I want is: >>>>>>>>> - 192.168.0.xxx, between 08:00-14:00, accessed the following >>>>>>>>> sites: ... >>>>>>>>> - www.facebook.com, between 08:00-14:00, was accessed by the >>>>>>>>> following >>>>>>>>> local IP-s: ... >>>>>>>>> >>>>>>>>> I don't need the: >>>>>>>>> - Host Traffic Stats >>>>>>>>> - Packet Statistics >>>>>>>>> - Protocol Distribution >>>>>>>>> - TCP/UDP Recently Used Ports >>>>>>>>> - IP Service Stats: Client Role >>>>>>>>> - TCP/UDP - Traffic on Other Ports >>>>>>>>> >>>>>>>>> How can I do all these? >>>>>>>>> >>>>>>>>> Istvan >>>>>>>>> >>>>>>>>> On 26.04.2010 17:12, Gary Gatten wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> Good call. One can also restrict the displayed protocols with >>>>>>>>>> -p, all remaining traffic will be displayed as "other" >>>>>>>>>> >>>>>>>>>> ----- Original Message ----- >>>>>>>>>> From: >>>>>>>>>> [email protected]<[email protected]> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> To: [email protected]<[email protected]>; >>>>>>>>>> [email protected]<[email protected]> >>>>>>>>>> Sent: Mon Apr 26 08:44:04 2010 >>>>>>>>>> Subject: Re: [Ntop] how to monitor http and https only >>>>>>>>>> >>>>>>>>>> Have you taken a look at the manpages for ntop? On a unix >>>>>>>>>> system, the "-B" switch followed by a pcap expression will give >>>>>>>>>> you want you want. >>>>>>>>>> >>>>>>>>>> e.g >>>>>>>>>> >>>>>>>>>> ntop -d -w 8080 -B "port 80 or 443" >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: [email protected] >>>>>>>>>> [mailto:[email protected]] On Behalf Of Istvan >>>>>>>>>> Köpe >>>>>>>>>> Sent: Monday, April 26, 2010 9:40 AM >>>>>>>>>> To: [email protected] >>>>>>>>>> Subject: [Ntop] how to monitor http and https only >>>>>>>>>> >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> I just installed ntop and it gives me much more information I >>>>>>>>>> need. I >>>>>>>>>> would like to see only the traffic on ports 80 and 443. >>>>>>>>>> How can I do that? >>>>>>>>>> >>>>>>>>>> Istvan >>>>>>>>>> _______________________________________________ >>>>>>>>>> Ntop mailing list >>>>>>>>>> [email protected] >>>>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>>>>> _______________________________________________ >>>>>>>>>> Ntop mailing list >>>>>>>>>> [email protected] >>>>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>>>>> _______________________________________________ >>>>>>>>>> Ntop mailing list >>>>>>>>>> [email protected] >>>>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Ntop mailing list >>>>>>>>> [email protected] >>>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>>>> _______________________________________________ >>>>>>>>> Ntop mailing list >>>>>>>>> [email protected] >>>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Ntop mailing list >>>>>>>> [email protected] >>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>>> _______________________________________________ >>>>>>>> Ntop mailing list >>>>>>>> [email protected] >>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> Ntop mailing list >>>>>>> [email protected] >>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>> _______________________________________________ >>>>>>> Ntop mailing list >>>>>>> [email protected] >>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> Ntop mailing list >>>>>> [email protected] >>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>> _______________________________________________ >>>>>> Ntop mailing list >>>>>> [email protected] >>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>>> >>>>>> >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> Ntop mailing list >>>>> [email protected] >>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>> _______________________________________________ >>>>> Ntop mailing list >>>>> [email protected] >>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>>> >>>>> >>>>> >>>> _______________________________________________ >>>> Ntop mailing list >>>> [email protected] >>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>> _______________________________________________ >>>> Ntop mailing list >>>> [email protected] >>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>> >>>> >>> >>> >> _______________________________________________ >> Ntop mailing list >> [email protected] >> http://listgateway.unipi.it/mailman/listinfo/ntop >> _______________________________________________ >> Ntop mailing list >> [email protected] >> http://listgateway.unipi.it/mailman/listinfo/ntop >> > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
