[squid-users] Squid with Squint REporting
Good morning everyone. I know this is a squid-only group, but I'm hoping someone can help, since it is related... I'm currently using Squid 3.18 and using Squint .92 for reporting. Weekly, monthly, and regen is working as needed. The problem I'm running into is the daily regen. Using both a cron job as well as running manually, I'm unable to get the daily reports to generate. There is a new field created in the webpage reports for the daily logs, but when viewing, it complains there is no data to display. I've checked the raw logs, and yes indeed, there is data there. Below are the errors I'm getting when running the reports. Other than called to early, the rest of the errors I get are the same when I run weekly and monthly regens. Any advice would be appreciated! -Begin copy/paste-- Input is read from LOGDIR=/var/log/squid3/Old Output is written to BASEDIR=/usr/local/httpd/htdocs/squint r...@nsa:/usr/local/bin# squint.cron.sh daily Generating report to /usr/local/httpd/htdocs/squint/all/daily from 20090622 to 20090624 main::writeusersitereports() called too early to check prototype at /usr/local/bin/squint.pl line 247. main::writeusersitereports() called too early to check prototype at /usr/local/bin/squint.pl line 341. Name main::messagelog used only once: possible typo at /usr/local/bin/squint.pl line 351. Name main::listlimit used only once: possible typo at /usr/local/bin/squint.pl line 168. Name main::peak used only once: possible typo at /usr/local/bin/squint.pl line 711. Name main::nametofilenamebasehref used only once: possible typo at /usr/local/bin/squint.pl line 570. Name main::basedom used only once: possible typo at /usr/local/bin/squint.pl line 351. Name main::basename used only once: possible typo at /usr/local/bin/squint.pl line 351. r...@nsa:/usr/local/bin# -End Copy- Thanks Dustin Dustin Hane IT Support Ph: 414-290-1128 Fx: 414-290-1515 500 W Oklahoma Ave Milwaukee, WI 53207 dust...@postalproducts.com
[squid-users] A Big Thank you..
Hey all..I've had a few questions in the past. Not many, but a few here and there. I've also been reading almost every post to learn more about squid... I just wanted to send out a big Thank you for all the knowledge everyone has passed on, and all the effort you've put into this to help other people you don't know..For free. A special mention to Amos..Who either does nothing but take care of people like me or just never sleeps. Keep up the great work!! Thanks Dustin Dustin Hane IT Support
RE: [squid-users] squid on windows domain users
What type of IP conflicts? Is yoru DHCP server handing down the same IP address to different machines? You may want to have a setup similar to this: IN your domain controller you have 2 subgroups under the COMPUTERS OU. Container 1 = Internet Access Allowed Place all the PCS you want to have IP access allowed in this Container. Container 2 = Not allowed Place all the PCs without access here. Direct your DHCP server to hand out a certain range to Container 1 and a different range to container 2. Within squid, set up a src acl for container 1 to allow. Follow? dear friends, i m from india is using a 2 Mbps Leased Line connection, distributing it through windows 2003 server with squid ip based filtering. frequently i suffer from problems like ip conflicts bcoz users who dnt hav internet facility track the ip on which internet is available changes them. what is the remedy to this. is there a solution like this. for all computers that need to hav internet facility, should be in domain of the system on which squid is installed only these will have internet facility no other computer on LAN can access internet, dsnt matter what its IP is. or there is a MAC based filtering available 4 windows in squid. whts ur opinions frnds? bye Now surf faster and smarter ! Check out the new Firefox 3 - Yahoo! Edition http://downloads.yahoo.com/in/firefox/
RE: [squid-users] squid on windows domain users
I'm sorry. I misunderstood that people were changing them. Best way to stop them from doing that, is to change the group policy settings in your domain controller to remove access to the network control panel for both users and local machines. Go to: User Configuration - Network - Network Connections Enable - Prohibit access to properties of components of LAN connection Enable - Prohibit TCP/IP advanced configuration Enable - Prohibit access to the Advanced Settings item on the Advanced Menu Enable - Prohibit access to properties of a LAN connection Enable - Prohibit access to the New Connection Wizard Then apply this GPO to your COMPUTERS OU and you'll be all set. Thanks Dustin -Original Message- From: Leonardo Carneiro [mailto:lscarne...@veltrac.com.br] Sent: Wednesday, April 29, 2009 9:45 AM To: Vicks Cc: squid-users@squid-cache.org Subject: Re: [squid-users] squid on windows domain users Hi Vicks, i'm from brazil and i hope i can help u. at the way i understood, users who don't have a full connection with the internet change their IPs to use the resources otherwise would be denied to them. you have tons of ways to prevent this. one way is to use the domain login (with a external program) to do the filtering. it will prevent ip conflict in your network. Vicks escreveu: dear friends, i m from india is using a 2 Mbps Leased Line connection, distributing it through windows 2003 server with squid ip based filtering. frequently i suffer from problems like ip conflicts bcoz users who dnt hav internet facility track the ip on which internet is available changes them. what is the remedy to this. is there a solution like this. for all computers that need to hav internet facility, should be in domain of the system on which squid is installed only these will have internet facility no other computer on LAN can access internet, dsnt matter what its IP is. or there is a MAC based filtering available 4 windows in squid. whts ur opinions frnds? bye Now surf faster and smarter ! Check out the new Firefox 3 - Yahoo! Edition http://downloads.yahoo.com/in/firefox/ -- *Leonardo de Souza Carneiro* *Veltrac - Tecnologia em Logística.* lscarne...@veltrac.com.br mailto:lscarne...@veltrac.com.br http://www.veltrac.com.br http://www.veltrac.com.br/ /Fone Com.: (43)2105-5600/ /Av. Higienópolis 1601 Ed. Eurocenter Sl. 803/ /Londrina- PR/ /Cep: 86015-010/
RE: [squid-users] Problems with IDENT lookup logging
1) you mention having questions but don't ask any. -- Well, one of them is..I have read that using LDAP lookup..When attempting to visit a blocked site, squid will challenge the authentication. Is this true? We're trying to keep this as transparent as possible. Will squid have any problems performing LDAP against a mail server? I have the mail server also set up as an LDAP server (it's an exchange2003 box), so, so long as I direct the requests under port 389 there shouldn't be a problem correct? Next question would be..Is there a better method to use than LDAP? NTLM possibly? 2) logging of authenticated username (LDAP) and loging of identity name (IDENT) are two separate things sometimes in Squid. Check the log format is showing what you want. --- I do have the log format set to record successfully the IDENT lookup. As you can see from the log..It does sometimes work and sometimes does not. I can include a much larger log file if anyone has the time to look it over. I do, but I can't discern any patterns.. 3) Ident is a rarely used (due to being insecure) method of identification. The re-write of auth for Squid-3 left a few problems in the way it works. Many of which are being resolved so recently the patches have not yet made it to 3.0 and some still waiting testing in bugzilla. If you need this kind of fix, please test the latest snapshots then get check bugzilla for any remaining issues. Again..I don't mind getting away from IDENT..It is a pain in the ass to get installed on all the client machines..But when I was first learning about squid, this is the path that was easiest for me (I had to learn linux first, then squid, then squint for reports, then IDENT for username logging..All in about a week).. So I just kind of stuck with it. We have until May 4th til this needs to go live. We, as you can see, are currently running and logging now so we can make sure the loads are all ok. So, any help before then would be awesome!! Thanks again guys!! Thanks Dustin Dustin Hane wrote: Hello all! I'm trying to get around having to do the LDAP or NTLM authentication schemas. It may be a lot easier, but I'm just not exactly sure how..So what I have done is this.. I pushed out via a GPO a script that will report the username to a text file. I then use windows IDent server (installed on all local boxes) to listen for when Squid makes an RFC 931 lookup request. The service responds with the username from the text file. Using Squid 3Stable7 on Unix..Exporting logs in default squid format.. I wouldn't have a problem using an LDAP server as I do have it set up..I just don't understand it and for some reason I can't wrap my head around the wiki for it and I have a few questions that aren't listed there..If someone has a few minutes that I could email my test config for it to, I would be eternally greatful! I just don't want to bog down the maillist with my stupidity. Works absolutely awesome 94% of the time..But occasionally I get the following. (usernames have been retracted for obvious reasons) A few things crop into y head reading your post: 1) you mention having questions but don't ask any. -- Well, one of them is..I have read that using LDAP lookup..When attempting to visit a blocked site, squid will challenge the authentication. Is this true? We're trying to keep this as transparent as possible. Will squid have any problems performing LDAP against a mail server? I have the mail server also set up as an LDAP server (it's an exchange2003 box), so, so long as I direct the requests under port 389 there shouldn't be a problem correct? Next question would be..Is there a better method to use than LDAP? NTLM possibly? 2) logging of authenticated username (LDAP) and loging of identity name (IDENT) are two separate things sometimes in Squid. Check the log format is showing what you want. 3) Ident is a rarely used (due to being insecure) method of identification. The re-write of auth for Squid-3 left a few problems in the way it works. Many of which are being resolved so recently the patches have not yet made it to 3.0 and some still waiting testing in bugzilla. If you need this kind of fix, please test the latest snapshots then get check bugzilla for any remaining issues. Amos ---Begin Logs--- 1240514814.201289 icm1512.postalproducts.com TCP_MISS/200 2347 GET http://www.bassind.com/images/bg_03.gif username DIRECT/65.198.197.121 image/gif 1240514814.578404 icm1512.postalproducts.com TCP_MISS/200 544 GET http://www.bassind.com/images/top_nav_bg.gif - DIRECT/65.198.197.121 image/gif 1240514814.613 1106 icm1512.postalproducts.com TCP_MISS/404 1561 GET http://www.bassind.com/images/main_top.gif - DIRECT/65.198.197.121 text/html 1240514814.673417 icm1512.postalproducts.com TCP_MISS/200 3994 GET http://www.bassind.com/prodimg/hometheatrehp.jpg username DIRECT
RE: [squid-users] Auto Detect Proxy in Browser, visiting users.
Also, you could do it the way I am running it..Or attempting to.. If you are on a windows domain (assuming you are as you're using LDAP or NTLM)..Use a Group policy object to push out the proxy. So long as all of your boxes are at least WIN2K you can do it in 4 minutes.. If you're not sure on how to do so, feel free to email me here or directly.. The rest is correct about LDAP and NTLM.. -Original Message- From: Amos Jeffries [mailto:squ...@treenet.co.nz] Sent: Wednesday, April 22, 2009 11:25 PM To: Chris Robertson Cc: squid-users@squid-cache.org Subject: Re: [squid-users] Auto Detect Proxy in Browser, visiting users. gavguinness wrote: Hi I'm new to Squid. New in the sense that this time yesterday, I didn't know what Squid was. I knew what I wanted to achieve though, and I've achieved most of this today using Squid and a few helpful online guides... To have users promted to authenticate when they start their browser (Check) To log their activity in a log file (Check) Not to have to install any software on the PC (Check) Specifically not to use any server based DB lookup authentication (check) The only problem is that I want all users to go through Squid, even visiting users. A lot of our guys are not going to want to manually enter Proxy settings each time they visit a site - I want it to be automatic. Similarly, not every user logs into our server(s), so I can't deploy a scrips or setting to the visiting computer as they simply connect to the WiFi, or Cabled network point. So basically, just connect up to the network, go on line and BAM, they have to authenticate. Just like in Starbucks! (But without the coffee or wifi charges!) I looked at transparent settings, but I gather this doesn't work with Authentication, so that's a no. Now i'm focussing on how to get the clients to auto detect the squid box. But I can't fathom how that's going to work. If the machines don't know it's there, how can squid make itself known to them? Ideally (and bear in mind my lack of knowledge at this stage) I would like to just have my DCHP tell the clients that the squid box is the default gateway and solve it that way, but again, I'm learning that the proxy doesn't work that way - it's not a router, right? Hope that makes sense, any help appreciated. But in the meantime, I'll get my head back in the manual! Cheers Look into WPAD (http://en.wikipedia.org/wiki/Web_Proxy_Autodiscovery_Protocol) or a captive portal like WiFiDog (http://en.wikipedia.org/wiki/WiFiDog_Captive_Portal) or the Squid session helper (check the archives). And definitely the relevant Squid FAQ entries: http://wiki.squid-cache.org/SquidFaq/ConfiguringBrowsers?highlight=%28WPAD%29 http://wiki.squid-cache.org/Technology/WPAD/DNS http://wiki.squid-cache.org/Technology/WPAD Here's the condensed version of what I have experienced with WPAD. It all assumes that the proxy settings have not been changed from the shipping default in the browsers. Using a Windows (98/2000/XP) machine and Internet Explorer, the DHCP option 252 is honored. DNS (wpad.domainname.com) is used in the absence of the DHCP option 252. Firefox (2 or 3) on a Windows (98/2000/XP) machine or OS X (10.4 for sure) the DHCP option 252 is ignored, DNS is used exclusively . Safari on Windows (98/2000/XP) or OS X ignores both DHCP and DNS and must be explicitly configured to use a statically defined PAC (http://en.wikipedia.org/wiki/Proxy_auto-config) file. My suggestion is to have a webserver assigned to http://wpad.yourdomain.tld that serves a PAC file when http://wpad.yourdomain.tld/wpad.dat OR http://wpad.yourdomain.tld/wpad.da is requested. This will (transparently) catch the majority of web browsers. For the rest, you should intercept outbound port 80 traffic and redirect it to a page that describes how to set their browser back to defaults (or how to set their browser to explicitly grab the PAC file). Chris
RE: [squid-users] Allow access to port 8080 from only one or two public IPs
Acl public_allow src public ip range here (ie. 64.64.64.0/8) Acl public_deny src public ip denied here Acl private_allow src private allowed here Acl private_deny src private ip denied here http_access allow public_allow http_access deny public_deny http_access allow private_allow http_access deny private_deny Everything is supposed to be in lower case..Outlook is trying to be helpful.. -Original Message- From: da...@davidwbrown.name [mailto:da...@davidwbrown.name] Sent: Wednesday, April 22, 2009 11:33 AM To: squid-users@squid-cache.org Subject: [squid-users] Allow access to port 8080 from only one or two public IPs Hello Amos and fellow Squid users, I am running Squid 3.0. I would like to block access to port 8080 accept for one or two public IPs and one or two internal class C IPs (192.168.1.1/24). Please advise if you have some definite caveats to share. Thanks, David. OS: CentOS 5.2 Squid: 3.0 port 8080: Tomcat 5.5 web application (a blog).
[squid-users] Problems with IDENT lookup logging
Hello all! I'm trying to get around having to do the LDAP or NTLM authentication schemas. It may be a lot easier, but I'm just not exactly sure how..So what I have done is this.. I pushed out via a GPO a script that will report the username to a text file. I then use windows IDent server (installed on all local boxes) to listen for when Squid makes an RFC 931 lookup request. The service responds with the username from the text file. Using Squid 3Stable7 on Unix..Exporting logs in default squid format.. I wouldn't have a problem using an LDAP server as I do have it set up..I just don't understand it and for some reason I can't wrap my head around the wiki for it and I have a few questions that aren't listed there..If someone has a few minutes that I could email my test config for it to, I would be eternally greatful! I just don't want to bog down the maillist with my stupidity. Works absolutely awesome 94% of the time..But occasionally I get the following. (usernames have been retracted for obvious reasons) ---Begin Logs--- 1240514814.201 289 icm1512.postalproducts.com TCP_MISS/200 2347 GET http://www.bassind.com/images/bg_03.gif username DIRECT/65.198.197.121 image/gif 1240514814.578 404 icm1512.postalproducts.com TCP_MISS/200 544 GET http://www.bassind.com/images/top_nav_bg.gif - DIRECT/65.198.197.121 image/gif 1240514814.613 1106 icm1512.postalproducts.com TCP_MISS/404 1561 GET http://www.bassind.com/images/main_top.gif - DIRECT/65.198.197.121 text/html 1240514814.673 417 icm1512.postalproducts.com TCP_MISS/200 3994 GET http://www.bassind.com/prodimg/hometheatrehp.jpg username DIRECT/65.198.197.121 image/jpeg 1240514824.037 356 icm1512.postalproducts.com TCP_MISS/404 1561 GET http://www.bassind.com/favicon.ico username DIRECT/65.198.197.121 text/html 1240514829.944 0 icm1338.postalproducts.com TCP_IMS_HIT/304 375 GET http://vendornet.americanhotel.com/colors/styles.css username NONE/- text/css 1240514829.946 0 icm1338.postalproducts.com TCP_IMS_HIT/304 391 GET http://vendornet.americanhotel.com/inc/main.js - NONE/- application/x-javascript 1240514829.969 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/colors/topB.gif username NONE/- image/gif 1240514830.000 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/Logo/AHLogo.gif - NONE/- image/gif 1240514830.004 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/colors/liteteal1x1.gif username NONE/- image/gif 1240514830.009 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/colors/exit.gif - NONE/- image/gif 1240514830.011 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/colors/topA.gif username NONE/- image/gif 1240514830.015 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/colors/leftReduce.gif - NONE/- image/gif 1240514830.021 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/colors/leftExpand.gif username NONE/- image/gif 1240514830.025 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/Colors/liteteal1x1.gif - NONE/- image/gif 1240514830.029 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/colors/arrow.gif username NONE/- image/gif 1240514830.034 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/colors/leftDiv.gif - NONE/- image/gif 1240514830.040 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/colors/arrowbl.gif username NONE/- image/gif 1240514830.049 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/colors/tealleft.gif - NONE/- image/gif 1240514830.050 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/colors/leftSpace.gif username NONE/- image/gif 1240514830.070 327 icm1338.postalproducts.com TCP_MISS/200 23941 POST http://vendornet.americanhotel.com/Index.asp jurgitad DIRECT/72.35.92.212 text/html 1240514830.080 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/images/powered.gif - NONE/- image/gif 1240514830.083 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/colors/teal1x1.gif username NONE/- image/gif 1240514830.093 0 icm1338.postalproducts.com TCP_IMS_HIT/304 376 GET http://vendornet.americanhotel.com/colors/recapright.gif username NONE/- image/gif 1240514832.457 107 icm1512.postalproducts.com TCP_MISS/200 1903 GET http://www.freightquote.com/images/qb_nav_account_on.gif username DIRECT/207.218.147.11 image/gif ---END LOGS Dustin Hane IT Support Ph: 414-290-1128 Fx: 414-290-1515 500 W Oklahoma Ave Milwaukee, WI 53207 dust
RE: [squid-users] allowedURL don't work
I'm trying to work with regex's and have a quick question in response to your response. Wouldn't you also be able to do just a url_regex -I pagesjuanes and allow that? That should theoretically work yes? If you are doing a url_allow and if you have the period infront of the domain, that allows anything from the tld.pagesjuanes.fr correct? ---Paste when i want access to www.pagejaunes.fr, he request a authentification ... i want no authentification and no limitation of surf. Anyone see where is my error ? the correct synthaxe are pagesjaunes.fr or .pagesjaunes.fr for *.pagesjaunes.fr ? The second option .pagesjaunes.fr will match http://pagesjaunes.fr, http://www.pagesjaunes.fr and any other hostname in front of pagesjaunes.fr. thanks jerome Chris End Paste -Original Message- From: crobert...@gci.net [mailto:crobert...@gci.net] Sent: Tuesday, April 21, 2009 12:59 PM To: squid-users@squid-cache.org Subject: Re: [squid-users] allowedURL don't work Phibee Network Operation Center wrote: Hi i have a new problems with my Squid Server (NTLM AD) My configuration: auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp auth_param ntlm children 15 auth_param ntlm keep_alive on auth_param basic program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic auth_param basic children 15 auth_param basic realm Squid proxy-caching web server auth_param basic credentialsttl 2 hours #external_acl_type AD_Group children=50 concurrency=50 %LOGIN /usr/lib/squid/wbinfo_group.pl external_acl_type AD_Group children=50 concurrency=50 ttl=1800 negative_ttl=900 %LOGIN /usr/lib/squid/wbinfo_group.pl cache_peer 127.0.0.1parent 80810 proxy-only no-query weight=100 connect-timeout=5 login=*:password ## ACL des droits d'accès acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 acl Lan src 10.0.0.0/8 # RFC1918 possible internal network acl Lan src 172.16.0.0/12 # RFC1918 possible internal network acl Lan src 192.168.0.0/16 # RFC1918 possible internal network ## ## ACL pour les sites web consultable sans authentification ## acl URL_Authorises dstdomain /etc/squid-ntlm/allowedURL http_access allow URL_Authorises Are you sure you don't want to add additional restrictions to the http_access allow (such as a limitation on the source IP, or something)? ## acl SSL_ports port 443 563 1 1494 2598 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 563 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT #http_access allow manager localhost http_access deny manager http_access deny !Safe_ports http_access deny CONNECT !SSL_ports ## # ACL pour definir les groupes AD autorisés a ce connecter ## acl AllowedADUsers external AD_Group /etc/squid-ntlm/allowedntgroups acl Winbind proxy_auth REQUIRED ## ## # ACL pour les Droits d'accès d'apres l'Active Directory ## # Droits d'accès d'apres l'Active Directory http_access allow AllowedADUsers http_access deny !AllowedADUsers http_access deny !Winbind These two deny lines are redundant, as everything is denied by the next line... ## http_access deny all ## # Parametre Systeme ## http_port 8080 hierarchy_stoplist cgi-bin ? cache_mem 16 MB #cache_dir ufs /var/spool/squid-ntlm 5000 16 256 cache_dir null /dev/null #logformat squid %ts.%03tu %6tr %a %Ss/%03Hs %st %rm %ru %un %Sh/%A %mt #logformat squidmime %ts.%03tu %6tr %a %Ss/%03Hs %st %rm %ru %un %Sh/%A %mt [%h] [%h] #logformat common %a %ui %un [%tl] %rm %ru HTTP/%rv %Hs %st %Ss:%Sh logformat combined %a %ui %un [%tl] %rm %ru HTTP/%rv %Hs %st %{Referer}h %{User-Agent}h %Ss:%Sh access_log /var/log/squid-ntlm/access.log squid cache_log /var/log/squid-ntlm/cache.log cache_store_log /var/log/squid-ntlm/store.log
[squid-users] Log Issues
-shockwave-flash --END WRONG ERROR LOGS-= I can see the difference, I just don't understand why it's happening. Any h= elp at all would be greatly appreciated!! Thanks Dustin Dustin Hane IT Support Ph: 414-290-1128 Fx: 414-290-1515 500 W Oklahoma Ave Milwaukee, WI 53207 dust...@postalproducts.com