Re: [squid-users] Path MTU discovery problem, question
On Mon, Aug 01, 2005 at 05:40:27PM -0700, john allspaw wrote: on the squids themselves, since it's production and I'm not 100% that it won't interrupt service. will it ??? is there anything else that folks might suggest to confirm that it is the pmtu problem ? TCP MSS clamping should reveal whether there's a PMTU-d issue. It looks like you're using linux - see the iptables manual. our squids sit behind a LB, with URL hash as it's balancing algorithm. thanks in advance, You should find out whether your LB can do MSS clamping too. It might be necessary if it spoofs the 3-way-handshake without honoring target server TCP parameters. -- Joshua Goodall as modern as tomorrow afternoon [EMAIL PROTECTED] - FW109
Re: [squid-users] LDAP_AUTH Problem Stable 3
On Wed, Jul 27, 2005 at 11:00:14AM +0200, Peter Schulz-Kraus wrote: Hello, I'm still using Squid 2.5 Stable3 under Fedora Core 1. (I will change it in about three months ;-) I'm using the LDAP_AUTH helper, but after each html- element (eg after each picture), I have to reauthenticate in my Browser. (IE6, Mozilla Firefox) Now I want to know, if I made a mistake in Squid.Conf or if it's a known bug of Stable 3. Here is the ACL- Part of squid.conf: (I hope, it's not too much) cache_peer cache001.xxx.de parent 8080 0 no-query login=PASS ^^^ Are you sure this peer isn't the culprit here, since you're forwarding the proxy authentication credentials? (do you need to do that?) - Joshua. -- Joshua Goodall as modern as tomorrow afternoon [EMAIL PROTECTED] - FW109
Re: [squid-users] SQUID Transparent Captive Portal w/ Authentication
On Tue, Jul 19, 2005 at 09:16:01AM +0200, Matus UHLAR - fantomas wrote: On 18.07 13:45, Lucia Di Occhi wrote: Has anyone implemented a captive portal registration/authentication system with squid in transparent mode? No. read the FAQ: http://www.squid-cache.org/Doc/FAQ/FAQ.html especially: http://www.squid-cache.org/Doc/FAQ/FAQ-17.html#ss17.16 You are too quick to dismiss the request. I have seen such systems implemented. He isn't talking about combining transparent proxying with 407 auth. If I was going to make another such system I'd probably just use a redirector. Squid already sends the client IP, and the redirector can just lookup a database to check session validity, redirecting to a local web server otherwise. Don't hotels do this routinely? At its simplest this can all be on the one box. Note that such systems are susceptible to IP address spoofing. A fully secured implementation would link DHCP-assigned IP address to switch port somehow, with the switch/router filtering appropriately. Doing this with 802.1q VLANs would also ensure a single broadcast domain per client, avoiding some security pitfalls of mutually distrusting systems on a single network. Alternatively, noting that the Cisco DHCP Relay Agent is documented to add per-lease static routes, you might then also use Cisco's Unicast RPF (reverse path forwarding), assuming the right hardware, network architecture, and IOS level. Depending on architecture and business requirements, you could just do a 802.1x deployment instead. Joshua. -- Joshua Goodall as modern as tomorrow afternoon [EMAIL PROTECTED] - FW109
Re: [squid-users] redirect based on User Agent ?
On Mon, Jul 11, 2005 at 01:04:02PM +0200, Len Kranendonk wrote: We're using multiple squids in http accelerator mode to speed up a high traffic website. At the moment we're working on a site for mobile / pda browsers. I'd like to redirect mobile users to an alternate URL. Is there a way to led squid handle the redirects based on User Agent ? Do you mean something like this? acl blackberry browser Blackberry acl windowsce browser Windows.CE acl symbian browser Symbian redirect_program /usr/local/bin/myredirector redirector_access allow blackberry redirector_access allow windowsce redirector_access allow symbian This page might help you with constructing regexes to use: http://www.zytrax.com/tech/web/mobile_ids.html Alternatively, you might find it more flexible to do this in the application. That'll allow you to handle many more special cases than Squid will. Looking at the variegated browser strings, I suspect there'll be plenty of gotchas. Regards Joshua. -- Joshua Goodall as modern as tomorrow afternoon [EMAIL PROTECTED] - FW109
Re: [squid-users] redirect based on User Agent ?
On Mon, Jul 11, 2005 at 02:46:25PM +0200, Len Kranendonk wrote: Thanks for the answer Joshua. I'm going to use WURLF ( http://wurfl.sourceforge.net/ ) for maintaining the list of devices. Do you know if it's possible to pass the User Agent variable to the redirector program ? You'd have to modify src/redirect.c. I have several such custom Squids in operation that pass various strings to redirectors (one even passes a cookie value). Regards Joshua.
Re: [squid-users] Concurrent Connection Limit
On Sun, Jul 10, 2005 at 02:04:36PM +0800, Jeffrey Ng wrote: Hi, I have problem with squid web accelerator on my site. My site is a photo sharing site like webshots. It has a pretty busy load, so I decided that squid may be able to sooth the load of my image server by caching some of the images. We have set everything up and it uses 1GB RAM. It was fine at first. But suddenly all the images stopped loading after 6 hours. I checked netstat and found that there are 1000 connections from outside. and squid stops responding whenever the connections hit that number. I am pretty sure that squid has a concurrent connection limit of 1000. How could I increase that limit? Any help is appreaciated. Thank you! Sounds like you're running out of filedescriptors. See http://www.squid-cache.org/Doc/FAQ/FAQ-11.html#ss11.4 - Joshua. -- Joshua Goodall as modern as tomorrow afternoon [EMAIL PROTECTED] - FW109
Re: [squid-users] Run 2 squids on same computer
On Tue, Jul 05, 2005 at 12:39:40AM +0200, Kinkie wrote: Squid doesn't scale well with the number of CPUs. So if you have a multiple-cpu box, the best way to exploit all the computing power is to run as many instances of Squid as the box has processors. I recommend also using your OS's CPU affinity selection. Prior to this, using RHAS3 on Hyperthreading Xeons, the scheduler was sometimes running two squids on the same physical unit. (e.g. taskset(8) on Linux) - Joshua. -- Joshua Goodall as modern as tomorrow afternoon [EMAIL PROTECTED] - FW109
Re: [squid-users] Development of SQL-based ACL?
On Thu, Jun 09, 2005 at 02:41:57PM +0200, Didde Brockman wrote: Hello users, I'm investigating the possibilities of using Squid in a large corporate environment. We have found the documentation to be rather complete, and after a period of evalutation two questions has come up and now I'm hoping you guys are able to help me out. 1. Are there any references available on _large_ projects incorporating Squid? Large being defined as roughly 100,000 - 200,000 clients 24/7. The numbers available on the site are outdated and seems to be focused on smaller solutions. I'm looking for some example hardware setups or basic numbers pertaining to the load of systems experiencing _heavy_ traffic. I've worked on multiple sites with requirements similar to yours. I'll just throw some numbers at you, then. You can easily do a couple of hundred requests per second - even with regex ACLs and authentication - on a single 3GHz Xeon. You don't say whether those are simultaneous clients or total seats, which makes a huge difference, nor whether you actually want to cache or just do this for filtering and access control. It'd help if you knew either the number of requests per second, or the bandwidth, or both. I've seen large clusters sized based on both those measures. Filtering can seriously damage your throughput - I've seen the same 3GHz Xeons reduced to a peak of 80req/sec (or about 4-5mbit/sec) through a heavyweight (but very full-featured) filtering subsystem. Above 8 servers I've observed serious degradation of the ICP protocol used for cache peering. So if you need more than that, or you might grow to more than that, either discontinue ICP (which will impact your hit ratio) or move to a hierarchy, with a lower layer doing authentication filtering plus an upper parent layer doing caching. Use source-IP-affinity loadbalancing as much as possible. There are some funny failure modes with other algorithms. Hardware load-balancers are usually a better choice than LVS, but LVS can do the job. Don't forget to make sure your high-availability tools (the LBs) are supported to the hilt - if you lose them, you lose the entire service. If you can, overprovision by at least 50% and have a 3/6/12-month capacity projection. 2. We would like to be able to administer ACL's through rules stored in a external database (SQL). Also, this would pair an IP with specific settings for a specific client, i.e. 10.0.0.122 is allowed to do this, but not that. Also, if client 10.0.0.122 requests a forbidden HTTP-resource its request should be redirected to a local page. Basically, these settings should be customisable for all users so they can acknowledge a forbidden resource, absorb the information on the local page (the target of the redirect) and then choose to continue on to their (forbidden) destination or not. You're going to need to develop a redirector. What you've described sounds remarkably similar to N2H2, but that might not be the kind of product that suits your business requirements. The redirector will probably then work in conjunction with a simple front-end to your policy database, which means policy must be changeable in real-time. So your database designs and internal protocols will also impact performance. I strongly advise use of 407 Proxy Authentication rather than per-IP controls, unless you've already solved the problem of authenticating identity by IP address (e.g. perhaps you're a dial-up ISP using RADIUS?) Have you thought about how you're going to manage banned URLs? There are many solutions. Trying to manage a list of blocked sites yourself is probably the worst one. For the second point, I'm assuming this would require us to contract a developer to make the necessary alterations the Squid's source code, so I'm also curious as to your experiences (if any) with known companies offering to do customisations on Squid. You shouldn't have to modify Squid itself, unless you want to do anything magical. On the other hand, if you do find something you need, get a really, really good Unix systems programmer. There are some lurking on the Squid mailing lists. I can recommend a couple that are based in Australia, if you mail me privately. If you decide to hire a company, be sure to check the technical background of their lead developers. You also need a good business analyst to develop the access rules framework, should you decide to develop your own rather than use one off-the-shelf. A bad system becomes horribly fine-grained and unusable very quickly. You also need to consider your reporting requirements when designing. Believe me, those logs become very large, very quickly. Get the balance right and you will easily push tens or hundreds of megabits through a cluster of Squids. Get it wrong and you've got hundreds of thousands of irate users on your hands. Best of luck :) -- Joshua Goodall as modern
Re: [squid-users] Bugs in IE digest proxy auth
Hi Henrik, On Fri, May 27, 2005 at 10:27:50AM +0200, Henrik Nordstrom wrote: Related question: What was the request-URI on the request line send by IE in the above? Was this escaped properly or where it using unescaped quote there as well? The request-URI was similarly unescaped. Uncertain. nonce reuse is quite dependent on the client having support for this. It may obviously be the case that IE do have support for nonce reuse but for some reason refuses to do so with Squid but it's very hard to know outside Microsoft. I wondered if there was some embrace-and-extended going on with Digest auth, but I've reproduced all of these bugs using ISA Server 2004 as well. Ethereal shows that it's all the same on the wire except for ISA using md5-sess. The only thing I can think of is to make sure there is persistent connections enabled. I could imagine that nonce reuse may be related to connection reuse in some clients. I have an experimental hack that turns digest auth into a per-connection authentication, a la NTLM. This cuts down on the excess 407 traffic. This ACL effectively downgrades the user to using basic proxy auth if using basic www auth. Interesting. So this worked around the problem for you? Kind of. The user sees the following: 1. User browses web normally with Digest proxy auth 2. User visits a site requiring 401 www-authentication 3. User is challenged and enters their 401 credentials 4. User is then re-challenged to enter their Basic proxy credentials 5. User then continues browsing, but for the remainder of that session IE is using basic proxy authentication for all requests. It's not an acceptable solution, because the password is now in the clear. Oddly, it doesn't happen with SSL. I'll work through this with MS. Notwithstanding the issues above, I have a six-figure userbase using Digest proxy auth successfully for 1200 requests/sec. At some point I'll seek authorisation to release our workarounds under the GPL. Joshua. -- Joshua Goodall as modern as tomorrow afternoon [EMAIL PROTECTED] - FW109
[squid-users] Bugs in IE digest proxy auth
Hi folks, I've observed three bugs in IE6's digest proxy authentication implemention. Is anyone able to confirm that it's Not Just Me, and whether any good workarounds are already out there? 1. Double-quote-marks in URLs. IE sends a digest-uri-value that's improperly escaped, as in (when visiting http://localhost/test.htmlfoo): Digest username=User Name, realm=Login, qop=auth, algorithm=MD5, uri=/test.htmlfoo, nonce=q3... vs Firefox which sends: Digest username=User Name, realm=Login, nonce=42yWQuDPGw/gebJ8, uri=/test.html%22foo, response=666... I've worked around this by means of a hack in authenticateDigestDecodeAuth. 2. Never reuses a nonce - always sends a request, gets a 407 challenge with a new nonce value, then reissues the request with the correct Proxy-Authorization header; this doubles the number of actual HTTP requests; is there a workaround? 3. IE gets confused when a website uses a 401 authentication challenge. In this case, it prompts for and sends the Authorization: header, but then repeatedly re-prompts the user for their proxy credentials. I've worked around this by inventing the following access list: acl msie-win-browser browser MSIE.*Windows acl basic-auth auth_regex Basic auth_param digest access deny msie-win-browser basic-auth auth_param digest access allow all Yes, I had to invent auth_regex and auth_param digest access. This ACL effectively downgrades the user to using basic proxy auth if using basic www auth. I'd especially like a solution for bug 3, because the workaround is ugly, insecure and expensive. I have a case open with Microsoft Partner Support right now. Oddly enough, IE5 on the Mac doesn't have any of these symptoms. Any takers? -- Joshua Goodall as modern as tomorrow afternoon [EMAIL PROTECTED] - FW109
