[squid-users] Squid as HTTP Compressor
Hi, Is it possible to use Squid 3.x as HTTP compressor like following configuration? HTTP Server -(LAN)- Reverse Squid -(WAN)- Forward Squid -(LAN)- Client If possible, please give me some clue of configuration of both Squids. Best regards,
[squid-users] Route large traffic through different gateway
Hello, I am using squid-3.1.11-1.el5 in a production server which acts as a gateway of our network. Every system is connected to the internet through the Squid proxy server. In my Squid configuration I am using reply_body_max_size 15 MB officelan to restrict download of large files. I am also using the following delay pool parameters to restrict Squid's overall bandwidth consumption. delay_pools 1 delay_class 1 1 delay_access 1 allow all delay_parameters 1 192000/192000 In my environment we have two ISP providers connected using two routers Link-1 and Link-2. Currently Squid uses Link-1 to connect to the internet. Currently if a download file size if more than 15 MB squid restricts the download as per the above configuration by my objective is to redirect requests through Link-2 if the download size is larger than 15 MB. While looking into the Squid mailing list archive I came across this post http://www.squid-cache.org/mail-archive/squid-users/200707/0678.html. I wen through the Squid documentation but I am not able to find how can a achieve it using Squid. Any help would be highly appreciated. Regards Supratik
Re: [squid-users] Route large traffic through different gateway
On Sun, 10 Apr 2011 21:04:59 +0530, Supratik Goswami wrote: Hello, I am using squid-3.1.11-1.el5 in a production server which acts as a gateway of our network. Every system is connected to the internet through the Squid proxy server. In my Squid configuration I am using reply_body_max_size 15 MB officelan to restrict download of large files. I am also using the following delay pool parameters to restrict Squid's overall bandwidth consumption. delay_pools 1 delay_class 1 1 delay_access 1 allow all delay_parameters 1 192000/192000 In my environment we have two ISP providers connected using two routers Link-1 and Link-2. Currently Squid uses Link-1 to connect to the internet. Currently if a download file size if more than 15 MB squid restricts the download as per the above configuration by my objective is to redirect requests through Link-2 if the download size is larger than 15 MB. While looking into the Squid mailing list archive I came across this post http://www.squid-cache.org/mail-archive/squid-users/200707/0678.html. I wen through the Squid documentation but I am not able to find how can a achieve it using Squid. Any help would be highly appreciated. You do it exactly as Henrik described in that post. tcp_outgoing_address sets the IP used to send requests, the OS does the policy work. Lookup policy routing in your OS networking documentation for how to set it up there. There is also the QoS equivalents (tcp_outgoing_tos and qos_flows) that can be used nowdays as alternatives to the outgoing IP. How to manage and configure QoS should also be in your OS networking documentation. Amos
Re: [squid-users] Squid as HTTP Compressor
On Sun, 10 Apr 2011 22:14:36 +0700, Lazuardi Nasution wrote: Hi, Is it possible to use Squid 3.x as HTTP compressor like following configuration? HTTP Server -(LAN)- Reverse Squid -(WAN)- Forward Squid -(LAN)- Client If possible, please give me some clue of configuration of both Squids. Best regards, Standard Squid configurations will do that. The two proxies are setup separately and unrelated. It is very common for an ISP forward proxy to be talking to a CDN reverse proxy elsewhere on the 'Net. Amos
[squid-users] Re: /dev/pf permission for squid 3.2.0.6 on openbsd 4.8
Hi reyk, many thanks for the reply. - revert /dev/pf to the old 0600 permissions reverted. Now it is set to default. pls see below. # ls -al /dev/pf crw--- 1 root wheel 73, 0 Apr 1 19:30 /dev/pf - recompile squid _without_ --enable-pf-transparent (disable it) recompiled without --enable-pf-transparent pls see squid configuration option ( Now, no --enable-pf-transparent option) # squid -v Squid Cache: Version 3.2.0.6 configure options: '--datadir=/usr/local/share/squid' '--enable-arp-acl' '--enable-basic-auth-helpers=NCSA' '--enable-digest-auth-helpers=password' '--enable-delay-pools' '--enable-external-acl-helpers=ip_user' '--enable-forw-via-db' '--enable-negotiate-auth-helpers=squid_kerb_auth' '--enable-removal-policies=lru' '--enable-ssl' '--enable-storeio=aufs' '--with-pthreads' '--localstatedir=/var/squid' '--prefix=/usr/local' '--sysconfdir=/etc/squid' '--mandir=/usr/local/man' '--infodir=/usr/local/info' --enable-ltdl-convenience - update your pf.conf to use divert-to instead of rdr-to updated. Pls see below. pass in log on $int_if proto tcp from $lan_net to any port 80 \ divert-to 127.0.0.1 port 3129 but, still now luck. any comments ? -- Thank you Indunil Jayasooriya
Re: [squid-users] Route large traffic through different gateway
@Amos I understand the policy routing and also checked with tcp_outgoing_address, but I am not able to figure out how can I use acl to filter the large traffic and use it with the tcp_outgoing_address. Currently with reply_body_max_size 15 MB officelan I am able to discard large downloads, but can you please tell me instead of discarding is there a way I can use the acl with tcp_outgoing_address to redirect the download through Link-2 (using policy routing) ? Regards Supratik On Mon, Apr 11, 2011 at 5:08 AM, Amos Jeffries squ...@treenet.co.nz wrote: On Sun, 10 Apr 2011 21:04:59 +0530, Supratik Goswami wrote: Hello, I am using squid-3.1.11-1.el5 in a production server which acts as a gateway of our network. Every system is connected to the internet through the Squid proxy server. In my Squid configuration I am using reply_body_max_size 15 MB officelan to restrict download of large files. I am also using the following delay pool parameters to restrict Squid's overall bandwidth consumption. delay_pools 1 delay_class 1 1 delay_access 1 allow all delay_parameters 1 192000/192000 In my environment we have two ISP providers connected using two routers Link-1 and Link-2. Currently Squid uses Link-1 to connect to the internet. Currently if a download file size if more than 15 MB squid restricts the download as per the above configuration by my objective is to redirect requests through Link-2 if the download size is larger than 15 MB. While looking into the Squid mailing list archive I came across this post http://www.squid-cache.org/mail-archive/squid-users/200707/0678.html. I wen through the Squid documentation but I am not able to find how can a achieve it using Squid. Any help would be highly appreciated. You do it exactly as Henrik described in that post. tcp_outgoing_address sets the IP used to send requests, the OS does the policy work. Lookup policy routing in your OS networking documentation for how to set it up there. There is also the QoS equivalents (tcp_outgoing_tos and qos_flows) that can be used nowdays as alternatives to the outgoing IP. How to manage and configure QoS should also be in your OS networking documentation. Amos
[squid-users] Re: /dev/pf permission for squid 3.2.0.6 on openbsd 4.8
updated. Pls see below. pass in log on $int_if proto tcp from $lan_net to any port 80 \ divert-to 127.0.0.1 port 3129 but, still now luck. any comments ? squid developer in squid mailing list said the below, Aha! so PF provides getsockname() now. That means it will require the ./configure --enable-ipfw-transparent option to Squid. so, I configured with ./configure --enable-ipfw-transparent here's the URL where We discussed http://www.mail-archive.com/squid-users@squid-cache.org/msg78526.html But, still no success. Where have I gone wrong? -- Thank you Indunil Jayasooriya
Re: [squid-users] Re: /dev/pf permission for squid 3.2.0.6 on openbsd 4.8
On 11/04/11 16:31, Indunil Jayasooriya wrote: Hi reyk, many thanks for the reply. - revert /dev/pf to the old 0600 permissions reverted. Now it is set to default. pls see below. # ls -al /dev/pf crw--- 1 root wheel 73, 0 Apr 1 19:30 /dev/pf - recompile squid _without_ --enable-pf-transparent (disable it) recompiled without --enable-pf-transparent pls see squid configuration option ( Now, no --enable-pf-transparent option) # squid -v Squid Cache: Version 3.2.0.6 configure options: '--datadir=/usr/local/share/squid' '--enable-arp-acl' '--enable-basic-auth-helpers=NCSA' '--enable-digest-auth-helpers=password' '--enable-delay-pools' '--enable-external-acl-helpers=ip_user' '--enable-forw-via-db' '--enable-negotiate-auth-helpers=squid_kerb_auth' '--enable-removal-policies=lru' '--enable-ssl' '--enable-storeio=aufs' '--with-pthreads' '--localstatedir=/var/squid' '--prefix=/usr/local' '--sysconfdir=/etc/squid' '--mandir=/usr/local/man' '--infodir=/usr/local/info' --enable-ltdl-convenience - update your pf.conf to use divert-to instead of rdr-to updated. Pls see below. pass in log on $int_if proto tcp from $lan_net to any port 80 \ divert-to 127.0.0.1 port 3129 but, still now luck. any comments ? 3.2 will not mark the traffic and do any of the special transparent traffic handling unless one of the NAT lookups functions returns true. Just relying on the default getsockname() is not sufficient to mark the traffic for special handling. Fortunately the ipfw NAT lookup does what the new PF version apparently needs. The --enable-ipfw-transparent should work as a temporary measure. I would like to fix this so --enable-pf-transparent properly detects and handles the version of PF available. Are you able to find out how I could do that please? Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.12 Beta testers wanted for 3.2.0.6
Re: [squid-users] Route large traffic through different gateway
On 11/04/11 16:52, Supratik Goswami wrote: @Amos I understand the policy routing and also checked with tcp_outgoing_address, but I am not able to figure out how can I use acl to filter the large traffic and use it with the tcp_outgoing_address. Currently with reply_body_max_size 15 MB officelan I am able to discard large downloads, but can you please tell me instead of discarding is there a way I can use the acl with tcp_outgoing_address to redirect the download through Link-2 (using policy routing) ? You must deduce the future size of data transfer and make whatever ACL tests tcp_outgoing_address needs based only on the *request* details. For things such as ... * POST or PUT the size is in the Content-Length header. * CONNECT you will have to make a judgement call, your logs should show some avg size trend. * HEAD will usually never reach 1 MB, though some can be amazingly high in the KB range. * GET is where it gets tricky. You will have to select based on the URL. The usual way is to just assume that multimedia, executable or archives file types are the big ones and the rest small. Perfection is not possible here, but the above generalizations will go along way towards biasing the traffic towards your desired large link. Sorry if I'm being a bit vague. The exact rules will depends very specifically on your clients and what parts of the web they visit. Well, thats how to do it in Squid at the point of outbound. The alternative if you want better reliability is ignoring Squid entirely and doing load balancing at the operating system packet level. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.12 Beta testers wanted for 3.2.0.6
Re: [squid-users] Re: /dev/pf permission for squid 3.2.0.6 on openbsd 4.8
On 11/04/11 17:11, Amos Jeffries wrote: On 11/04/11 16:31, Indunil Jayasooriya wrote: Hi reyk, many thanks for the reply. - revert /dev/pf to the old 0600 permissions reverted. Now it is set to default. pls see below. # ls -al /dev/pf crw--- 1 root wheel 73, 0 Apr 1 19:30 /dev/pf - recompile squid _without_ --enable-pf-transparent (disable it) recompiled without --enable-pf-transparent pls see squid configuration option ( Now, no --enable-pf-transparent option) # squid -v Squid Cache: Version 3.2.0.6 configure options: '--datadir=/usr/local/share/squid' '--enable-arp-acl' '--enable-basic-auth-helpers=NCSA' '--enable-digest-auth-helpers=password' '--enable-delay-pools' '--enable-external-acl-helpers=ip_user' '--enable-forw-via-db' '--enable-negotiate-auth-helpers=squid_kerb_auth' '--enable-removal-policies=lru' '--enable-ssl' '--enable-storeio=aufs' '--with-pthreads' '--localstatedir=/var/squid' '--prefix=/usr/local' '--sysconfdir=/etc/squid' '--mandir=/usr/local/man' '--infodir=/usr/local/info' --enable-ltdl-convenience - update your pf.conf to use divert-to instead of rdr-to updated. Pls see below. pass in log on $int_if proto tcp from $lan_net to any port 80 \ divert-to 127.0.0.1 port 3129 but, still now luck. any comments ? 3.2 will not mark the traffic and do any of the special transparent traffic handling unless one of the NAT lookups functions returns true. Just relying on the default getsockname() is not sufficient to mark the traffic for special handling. Fortunately the ipfw NAT lookup does what the new PF version apparently needs. The --enable-ipfw-transparent should work as a temporary measure. I would like to fix this so --enable-pf-transparent properly detects and handles the version of PF available. Are you able to find out how I could do that please? Oops. Nevermind. I see your other mail now. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.12 Beta testers wanted for 3.2.0.6
