RE: [squid-users] HowDo I: Get Date stamp on the access.log
I was looking at a similar issue today and needed to convert the epoch time into a human readable time on the command line. With FreeBSD you can use date -r epoch time i.e. # date -r 1154520379 Wed Aug 2 13:06:19 BST 2006 # I imagine you can do this with other OSs using their specific syntax. Paul. -Original Message- From: John Walubengo [mailto:[EMAIL PROTECTED] Sent: 10 July 2006 06:48 To: Geoff Varney Cc: squid-users@squid-cache.org Subject: RE: [squid-users] HowDo I: Get Date stamp on the access.log Thanx Geoff. U have given me exactly what I needed; a step by step solution aka the 'dummy's version' ;-). thanx alot. walu. --- Geoff Varney [EMAIL PROTECTED] wrote: John, Here are the files if you're interested. Geoff -Original Message- From: John Walubengo [mailto:[EMAIL PROTECTED] Sent: Friday, July 07, 2006 1:39 AM To: Guido Serassio; Laurent Marc 00 Cc: squid-users@squid-cache.org Subject: Re: [squid-users] HowDo I: Get Date stamp on the access.log Below is my sample access.log data: 1152258629.089 23364 172.16.32.80 TCP_MISS/200 4625 GET http://www.pcreview.co.uk/template/vbulletin.css student DIRECT/70.86.33.18 text/css 1152258629.261 2792 172.16.32.80 TCP_MISS/200 2570 GET http://www.htmlgoodies.com/css/starlight/star-light.css student DIRECT/63.236.73.67 text/css It is basically the default log; how can i get it to show the date:time stamp so that I get to know WHEN the users accessed these sites? walu. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
RE: [squid-users] Memory Error when using large acl files
Yes that is what I was doing Now I have converted to a dstdomain acl and everything is fine. Squid is using 37MB memory on a PIII/1GHz with 512MB RAM. When I load the current squid configuration from cache manager, squid shoots up to 99% CPU load and then the browser crashes. I have to send a shutdown signal to regain normality. I guess the browser just cannot handle displaying 550,000 entries. It is a shame because I find the config display very useful Thanks for your help Paul -Original Message- From: Henrik Nordstrom [mailto:[EMAIL PROTECTED] Sent: 28 February 2006 21:36 To: Paul Mattingly Cc: squid-users@squid-cache.org Subject: RE: [squid-users] Memory Error when using large acl files tis 2006-02-28 klockan 15:54 + skrev Paul Mattingly: Why does squid's memory usage increase by nearly 320MB when the file is only 9MB? I would guess because you are using regex acl, and each line gets compiled into a compiled regex internally to speed up the processing. A dstdomain ACL of 600K entries or 9MB uses 37MB of memory on 64-bit platforms or 23MB of memory on 32-bit platforms in my tests. Startup time for parsing this dstdomain acl was about 15-20 seconds. Which of the redirectors/plug-ins are best for managing large blacklists if this way just won't work on this scale? The Squid dstdomain ACL is about the fastest you can find at the moment. The SquidGuard url ACL is the most flexible for more detailed matches beyond only the hostname, but overhead of using a redirector is very significant. the regex type acls is bad performers in both. Not much which can be done about that as regex have no structure. SquidGuard has one nice feature in that it can use db files to avoid building the complete index in memory on startup. And due to SquidGuard being a redirector this also saves considerably amount of memory compared to each copy of SquidGuard building it's own in-memory index.. Regards Henrik
RE: [squid-users] Memory Error when using large acl files
I am also having problems with this, but found if you use acl porn_domains dstdomain /usr/local/squid/blacklists/porn/domains and add a period/full stop in front of the domain in the file, it will block that domain and any subdomains of it for example .google.com blocks Google.com www.google.com images.google.com groups.google.com fish-soup.google.com and so on. I believe this is as useful as dstdom_regex. But just using a 9 meg text file as you are, I have not managed to get squid running in a stable condition or even reconfiguring. My two questions are Why does squid's memory usage increase by nearly 320MB when the file is only 9MB? Which of the redirectors/plug-ins are best for managing large blacklists if this way just won't work on this scale? Thanks Paul -Original Message- From: Carsten Jensen [mailto:[EMAIL PROTECTED] Sent: 28 February 2006 10:19 To: squid-users@squid-cache.org Subject: RE: [squid-users] Memory Error when using large acl files you can see my config here below as to why.. well I don't want my users to surf porn. the dstdomain_regex because the file contains fx sex.com but the homepage can be www.sex.com If I use .sex.com in the file domains (path below) with the acl dstdomain I won't even be able to access www.google.com (where the browser then shows page not found) acl porn_domains dstdomain_regex /usr/local/squid/blacklists/porn/domains http_access deny all porn_domains best regards Carsten Jensen mån 2006-02-27 klockan 14:30 +0100 skrev Carsten Jensen: Hello. I have this problem that I have a large file in which I have a lot of domainnames for which I want to block all. The file is around 9 megs. What kind of acl are you using, and why that kind of acl? aclParseRegexList: Invalid regular expression 'domain.tld': out of memory Looks to me you are esing regex based acls. why? Regards Henrik
[squid-users] squid and AD configuration guidelines
Hi I have spent the last few months getting Squid to work seamlessly in a Windows 2003 AD environment. Being an MCSE I had very little *NIX knowledge but I had to try Squid out as ISA was not an option. I would like to share my configuration with others so hopefully I can provide the same help I received. I make no guarantees, this is not a complete how-to, it's just what I did to get things running in my particular environment with the software versions specified. There is much improvement to be made and a great deal for me to learn, but this is working just fine at the moment. Please please try this in a test environment first. I was dumb enough not to do so and ended up killing a production DC when trying to join the squid machine to the domain. An error in smb.conf over-wrote the DC's computer account in AD! Oops. I just treated the situation as if the DC had an unrecoverable hardware failure. Following an MS article, I removed the DC from AD by hand and rebuilt it under a new name. I felt this was the only way to be sure, and everything is back to normal now! Won't be forgetting that in a hurry; what doesn't kill you (or the network) can only make you stronger! :-P So here we go : - ) Hardware HP Netserver LC 2000 U3 Pentium III/1000Mhz 512MB RAM 1x18GB SCSI drive 2x36GB SCSI drive I created two RADI0 volumes, one with one disk and one with two disks. This favours performance over fault-tolerance. Software FreeBSD 6.0-RELEASE http://www.freebsd.org/ Squid 2.5 STABLE12 http://www.squid-cache.org/ Samba 3.0.21a http://www.samba.org/ Windows 2003 SP1 Active Directory environment Operating System setup ¯¯ FreeBSD was loaded with standard partitions (/ /var /usr and swaps) on the first disk. I created one large partition mounted /disk1 for the cache on the second disk. The OpenLDAP libraries from the ports collection are required to communicate with AD. You can use sysinstall during installation or later to install this. Configure, Packages, Select Media, Net, openldap-client-2.2.27. Next came the user and group accounts to run squid under. These were called proc_squid and grp_squid and created in the normal way as per the handbook. To allow use of the cache manager, Apache 1.3 was installed from /usr/ports/www/apache13/ Samba ¯ Samba is required to facilitate transparent NTLM authentication. Only winbind ends up running so it seems overkill to install the whole package. Follow the installation instructions and make sure to add --with-winbind --with-ads when you run the configure script. If you get errors that relate to LDAP not being installed you can specify where the libs are like this. I imagine this will vary between OSs, this is what FreeBSD required. --libdir=/usr/local/lib/ --includedir=/usr/local/include/ You can use the smb.conf at the bottom of this page as a guideline for your own to get Samba running. An excellent FAQ is located at http://www.squid-cache.org/Doc/FAQ/FAQ-23.html which describes testing procedures. The only program I used from Samba was ntlm-auth which in turn relies on winbindd to function. This will authenticate the user transparently and pass the details of the account to Squid via the external helpers setup. Rather more info than you need (!) can be found here http://us1.samba.org/samba/docs/man/Samba-HOWTO-Collection/winbind.html There is also an excellent guide regarding Samba and squid here. http://pserver.samba.org/samba/docs/man/Samba-Guide/DomApps.html The squid machine has to be joined to the AD domain, and you can do this with the following command /usr/local/samba/bin/net ads join -U administrator%password While you are in AD U+C checking the account is OK, you might as well create the account which the LDAP program will use to authenticate. Just a regular user account with no access will do just fine. Use the credentials when constructing the squid_ldap_group command line as detailed below. There is a section in one of the FAQs about using a cron job to cycle the computer account password every so often. It's not obvious whether this is required or not, I certainly haven't had to do it yet. However, if the authentication should break down unexpectedly, it's one of the first things I will look at! I encountered various different errors here and a summary follows BH NT_STATUS_ACCESS_DENIED [2005/12/14 14:12:09, 0] utils/ntlm_auth.c:winbind_pw_check(439) Login for user [EMAIL PROTECTED] failed due to [winbind client not authorized to use winbindd_pam_auth_crap. Ensure permissions on /var/db/samba/winbindd_privileged are set correctly.] The permissions on /var/db/samba/winbindd_privileged is not set correctly The directory I had to check was /usr/local/samba/var/locks/winbindd_privileged/ User: root or cache_effective_user Group: cache_effective_group
RE: [squid-users] Save clients password
If you can configure squid to use NTLM, this authentication process happens in the background with no user intervention which may be a better option. Is your ldap program talking to active directory or some other ldap directory? -Original Message- From: Franco, Battista [mailto:[EMAIL PROTECTED] Sent: 22 February 2006 12:12 To: squid-users@squid-cache.org Subject: [squid-users] Save clients password Hi I use squid ldap users authentication. From my client PCs every time I start IE I need to insert username and password. Is it possible to configure squid user and password popup with a checkbox to permit to save password? So next time I'll not retype password.
