[squid-users] How to create an ACL matching patterns from an URL
Is there a way to create an acl in such a way that I can only block few pages from that domain ? Example: If there is a domain named example.com, I will allow all pages except when it matches the following in the URL. example.com/home.php example.com/home.php#!/profile.php Regards Supratik
Re: [squid-users] How to create an ACL matching patterns from an URL
Hello Amos, I checked the documentation, it is working fine when I match only the domain. acl forbiddenURLs url_regex -i /etc/squid/forbiddenURL.txt http_access deny forbiddenURLs Any domain name I put in the forbiddenURL.txt is working fine. for example: .example.com can block everything for that domain. It is not working when the request is redirected to the HTTPS page (home.php) after the login page of that domain when I modify the expression to include .example.com/home.php. Is there any way I can validate my acl statements against the URL ? (I want to know if I am doing it correctly). Regards Supratik On Thu, Jul 14, 2011 at 6:07 PM, Amos Jeffries squ...@treenet.co.nz wrote: On 15/07/11 00:17, Supratik Goswami wrote: Is there a way to create an acl in such a way that I can only block few pages from that domain ? Example: If there is a domain named example.com, I will allow all pages except when it matches the following in the URL. example.com/home.php example.com/home.php#!/profile.php Regards Supratik Introducing the access control lists (ACL) documentation: http://wiki.squid-cache.org/SquidFaq/SquidAcl http://www.squid-cache.org/Doc/config/acl/ http://www.squid-cache.org/Doc/config/http_access/ NOTE: the #... part is called a page fragment and is completely internal to the web browser. The second one is never seen by Squid. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.14 Beta testers wanted for 3.2.0.9
Re: [squid-users] How to create an ACL matching patterns from an URL
Hello Amos, Thanks for the information. Regards Supratik On Thu, Jul 14, 2011 at 7:11 PM, Amos Jeffries squ...@treenet.co.nz wrote: On 15/07/11 01:26, Supratik Goswami wrote: Hello Amos, I checked the documentation, it is working fine when I match only the domain. acl forbiddenURLs url_regex -i /etc/squid/forbiddenURL.txt http_access deny forbiddenURLs Any domain name I put in the forbiddenURL.txt is working fine. for example: .example.com can block everything for that domain. It is not working when the request is redirected to the HTTPS page (home.php) after the login page of that domain when I modify the expression to include .example.com/home.php. Is there any way I can validate my acl statements against the URL ? (I want to know if I am doing it correctly). HTTP_S_ is a different problem entirely. Squid never sees the /path piece of the URL their. That is buried in the encrypted area. All Squid sees for https:// is the host and port which the encrypted data is to be sent. Example: CONNECT domain:port HTTP/1.1 Host: domain ... binary encrypted data... Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.14 Beta testers wanted for 3.2.0.9
Re: [squid-users] Filtering based on content size.
@Amos Thanks for the information. There is one confusion still in my mind. How reply_body_max_size is able detect it ? In the Squid documentation it says: This size is checked twice. First when we get the reply headers, we check the content-length value. If the content length value exists and is larger than the allowed size, the request is denied and the user receives an error message that says the request or reply is too large. If there is no content-length, and the reply size exceeds this limit, the client's connection is just closed and they will receive a partial reply. So, I think if something similar to reply_body_max_size or any workaround is present which uses reply_body_max_size directive then the issue could be easily resolved. Regards Supratik On Thu, Apr 28, 2011 at 6:25 PM, Amos Jeffries squ...@treenet.co.nz wrote: On 28/04/11 17:56, Supratik Goswami wrote: @Amos Thanks for your reply. Currently I am using acl to filter file extension .exe, .iso, .zip and using with tcp_outgoing_address I am able to change the source IP and it is working fine with source based routing. I want to filter by size (Ex. 15MB) which I am unable to do it using ACL. On the other hand reply_body_max_size does filtering based on size but I am not able to use it as per my requirement. Is it a limitation of Squid ? It is a limitation of this universe. In HTTP the size of a reply is at best-case mentioned in the headers. At which point the connection is already open and the request has already been sent out. Worst-case the size is not known until the very last byte has been received. Please let me know if there is any way to resolve this issue. There is none which always works. The file extension guessing is nearly close to as Squid can get. Beyond it you are doing things like manually measuring individual sites and adding particular domains to the outgoing address selection. Tedious never ending work. My advice is to go for load balancing at the operating system level. Load balancing works much simpler down there and you can do things like NAT the outgoing address of new connections based on the proportion of traffic each NIC has received so far (bumpy in the short term, but averages out over many connections). Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.12 Beta testers wanted for 3.2.0.7 and 3.1.12.1
[squid-users] Filtering based on content size.
The reply_body_max_size directive prevents users from downloading very
large files.
The following configuration in Squid only allows download of size 15MB
from IP range
mentioned in the acl officelan. If the size is more it simply
restricts the user with an
error message.
acl officelan src 192.168.1.0-192.168.1.54
reply_body_max_size 15 MB officelan
Is there any way I can do something else instead of showing the error
page to user
when the size is more than 15MB.
Something similar to the following:
if ( reply_body_max_size 15MB )
{
// do this;
}
else
{
// do this;
}
Regards
Supratik
Re: [squid-users] Filtering based on content size.
@Amos Thanks for your reply. Currently I am using acl to filter file extension .exe, .iso, .zip and using with tcp_outgoing_address I am able to change the source IP and it is working fine with source based routing. I want to filter by size (Ex. 15MB) which I am unable to do it using ACL. On the other hand reply_body_max_size does filtering based on size but I am not able to use it as per my requirement. Is it a limitation of Squid ? Please let me know if there is any way to resolve this issue. Regards Supratik On Thu, Apr 28, 2011 at 5:01 AM, Amos Jeffries squ...@treenet.co.nz wrote: On Wed, 27 Apr 2011 15:36:52 +0530, Supratik Goswami wrote: The reply_body_max_size directive prevents users from downloading very large files. The following configuration in Squid only allows download of size 15MB from IP range mentioned in the acl officelan. If the size is more it simply restricts the user with an error message. What would you have it do? Amos
Re: [squid-users] Route large traffic through different gateway
@Amos, Sorry for the late reply. I experimented a bit during this time. I decided to first forward all traffic from a single system to a different gateway. Once this works fine I will go for filtering based on download size. So, to allow all requests from a different gateway link I did the following. In my Squid configuration file I added the following lines. Here I want to allow all requests to from IP 192.168.1.226 to be directed through the second gateway link2. acl ext_filtering src 192.168.1.226 tcp_outgoing_address 192.168.1.250 ext_filtering Now for the second gateway I created a routing table link2 and added the following default route and routing rules. ip rule add from 192.168.1.250 table link2 ip route add default via 192.168.1.253 dev eth1 table link2 ip route flush cache Here 192.168.1.253 is the second gateway. Now, when I test this by running the command elinks --dump whatismyipaddress.com |head -20 from the system with IP 192.168.1.226 it works sometimes and sometimes it doesn't. Also for some sites I am getting Access Denied error. I have been pulling my hairs out on this issue for a pretty long time but unable to get a proper solution. Can you please tell me if I am going wrong anywhere ? On Mon, Apr 11, 2011 at 10:59 AM, Amos Jeffries squ...@treenet.co.nz wrote: On 11/04/11 16:52, Supratik Goswami wrote: @Amos I understand the policy routing and also checked with tcp_outgoing_address, but I am not able to figure out how can I use acl to filter the large traffic and use it with the tcp_outgoing_address. Currently with reply_body_max_size 15 MB officelan I am able to discard large downloads, but can you please tell me instead of discarding is there a way I can use the acl with tcp_outgoing_address to redirect the download through Link-2 (using policy routing) ? You must deduce the future size of data transfer and make whatever ACL tests tcp_outgoing_address needs based only on the *request* details. For things such as ... * POST or PUT the size is in the Content-Length header. * CONNECT you will have to make a judgement call, your logs should show some avg size trend. * HEAD will usually never reach 1 MB, though some can be amazingly high in the KB range. * GET is where it gets tricky. You will have to select based on the URL. The usual way is to just assume that multimedia, executable or archives file types are the big ones and the rest small. Perfection is not possible here, but the above generalizations will go along way towards biasing the traffic towards your desired large link. Sorry if I'm being a bit vague. The exact rules will depends very specifically on your clients and what parts of the web they visit. Well, thats how to do it in Squid at the point of outbound. The alternative if you want better reliability is ignoring Squid entirely and doing load balancing at the operating system packet level. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.12 Beta testers wanted for 3.2.0.6
[squid-users] Route large traffic through different gateway
Hello, I am using squid-3.1.11-1.el5 in a production server which acts as a gateway of our network. Every system is connected to the internet through the Squid proxy server. In my Squid configuration I am using reply_body_max_size 15 MB officelan to restrict download of large files. I am also using the following delay pool parameters to restrict Squid's overall bandwidth consumption. delay_pools 1 delay_class 1 1 delay_access 1 allow all delay_parameters 1 192000/192000 In my environment we have two ISP providers connected using two routers Link-1 and Link-2. Currently Squid uses Link-1 to connect to the internet. Currently if a download file size if more than 15 MB squid restricts the download as per the above configuration by my objective is to redirect requests through Link-2 if the download size is larger than 15 MB. While looking into the Squid mailing list archive I came across this post http://www.squid-cache.org/mail-archive/squid-users/200707/0678.html. I wen through the Squid documentation but I am not able to find how can a achieve it using Squid. Any help would be highly appreciated. Regards Supratik
Re: [squid-users] Route large traffic through different gateway
@Amos I understand the policy routing and also checked with tcp_outgoing_address, but I am not able to figure out how can I use acl to filter the large traffic and use it with the tcp_outgoing_address. Currently with reply_body_max_size 15 MB officelan I am able to discard large downloads, but can you please tell me instead of discarding is there a way I can use the acl with tcp_outgoing_address to redirect the download through Link-2 (using policy routing) ? Regards Supratik On Mon, Apr 11, 2011 at 5:08 AM, Amos Jeffries squ...@treenet.co.nz wrote: On Sun, 10 Apr 2011 21:04:59 +0530, Supratik Goswami wrote: Hello, I am using squid-3.1.11-1.el5 in a production server which acts as a gateway of our network. Every system is connected to the internet through the Squid proxy server. In my Squid configuration I am using reply_body_max_size 15 MB officelan to restrict download of large files. I am also using the following delay pool parameters to restrict Squid's overall bandwidth consumption. delay_pools 1 delay_class 1 1 delay_access 1 allow all delay_parameters 1 192000/192000 In my environment we have two ISP providers connected using two routers Link-1 and Link-2. Currently Squid uses Link-1 to connect to the internet. Currently if a download file size if more than 15 MB squid restricts the download as per the above configuration by my objective is to redirect requests through Link-2 if the download size is larger than 15 MB. While looking into the Squid mailing list archive I came across this post http://www.squid-cache.org/mail-archive/squid-users/200707/0678.html. I wen through the Squid documentation but I am not able to find how can a achieve it using Squid. Any help would be highly appreciated. You do it exactly as Henrik described in that post. tcp_outgoing_address sets the IP used to send requests, the OS does the policy work. Lookup policy routing in your OS networking documentation for how to set it up there. There is also the QoS equivalents (tcp_outgoing_tos and qos_flows) that can be used nowdays as alternatives to the outgoing IP. How to manage and configure QoS should also be in your OS networking documentation. Amos
