Re: [squid-users] load balancing and site failover
On Thu, 2015-03-26 at 13:53 +1300, Amos Jeffries wrote: On 26/03/2015 10:26 a.m., Brendan Kearney wrote: On Wed, 2015-03-25 at 15:03 +1300, Amos Jeffries wrote: On 25/03/2015 9:55 a.m., brendan kearney wrote: Was not sure if bugzilla was used for mailing list issues. If you would like me to open one, I will but it looks like the list is working again. Bugzilla is used, list bugs under the project services product. As for your query... On Mar 24, 2015 2:25 PM, Brendan Kearney wrote: On Tue, 2015-03-24 at 10:18 -0400, Brendan Kearney wrote: while load balancing is not a requirement in a proxy environment, it does afford a great deal of functionality, scaling and fault tolerance in one. several if not many on this list probably employ them for their proxies and likely other technologies, but they are not all created equal. i recently looked to see if a specific feature was in HAProxy. i was looking to see if HAProxy could reply to a new connection with a RST packet if no pool member was available. the idea behind this is, if all of the proxies are not passing the service check and are marked down by the load balancer, the reply of a RST in the TCP handshake (i.e. SYN - RST, not SYN - SYN/ACK - ACK) tells the browser to failover to the next proxy assigned by the PAC file. where i work, we have this configuration working. the load balancers are configured with the option to send a reset when no proxy is available in the pool. the PAC file assigns all 4 of the proxy VIPs in a specific order based on which proxy VIP is assigned as the primary. In every case, if the primary VIP does not have an available pool member, the browser fails over to the next in the list. failover would happen again, if the secondary VIP replies with a RST during the connection establishing. the process repeats until a TCP connection establishes or all proxies assigned have been exhausted. the browser will use the proxy VIP that it successfully connects to, for the duration of the session. once the browser is closed and reopened, the evaluation of the PAC file occurs again, and the process starts anew. plug-ins such as Proxy Selector are the exception to this, and can be used to reevaluate a PAC file by selecting it for use. we have used this configuration several times, when we found an ISP link was flapping or some other issue more global in nature than just the proxies was affecting our egress and internet access. i can attest to the solution as working and elegantly handling site wide failures. being that the solutions where i work are proprietary commercial products, i wanted to find an open source product that does this. i have been a long time user of HAProxy, and have recommended it for others here, but sadly they cannot perform this function. per their mailing list, they use the network stack of the OS for connection establishment and cannot cause a RST to be sent to the client during a TCP handshake if no pool member is available. they suggested an external helper that manipulates IPTables rules based on a pool member being available. they do not feel that a feature like this belongs in a layer 4/7 reverse proxy application. They are right. HTTP != TCP. i didnt confuse that detail. it was unknown to me that HAProxy could not tie layer 7 status to layer 3/4 actions. the decisions they made and how they architected the app is why they cannot do this, not that it is technically impossible to do it. i may be spoiled because i work with equipment that can do this for me. In particular TCP depends on routers having a full routing map of the entire Internet (provided by BGP) and deciding the best upstream hop based on that global info. Clients have one (and only one) upstream router for each server they want to connect to. i will contest this. my router does not need a full BGP map to route traffic locally on my LAN or remotely out its WAN interface. hell, it does not even run BGP, and i can still get to the intarwebs, no problem. it too, only has one upstream router / default route. Then your router has more in common with proxy than usual. Its operating with a next-hop packet relay model (OSPF? MPLS?) rather than an end-to-end model (BGP with RIB/FIB). DOCSIS 2 - ethernet on the WAN side and locally connected on the LAN side. :D oh, and a static route pointing a /24 for vpn traffic to a specific device. In HTTP each proxy (aka router) performs independent upstream connection attempts, failover, and verifies it worked before responding to the client with a final response. Each proxy only has enough detail to check its upstream(s). Each proxy can connect to any server (subject to ACLs). how are you comparing a HTTP proxy (a layer 7 application) to a router (a layer 3 device)? routers route traffic and proxies proxy traffic. while, routers
Re: [squid-users] load balancing and site failover
On Wed, 2015-03-25 at 15:03 +1300, Amos Jeffries wrote: On 25/03/2015 9:55 a.m., brendan kearney wrote: Was not sure if bugzilla was used for mailing list issues. If you would like me to open one, I will but it looks like the list is working again. Bugzilla is used, list bugs under the project services product. As for your query... On Mar 24, 2015 2:25 PM, Brendan Kearney wrote: On Tue, 2015-03-24 at 10:18 -0400, Brendan Kearney wrote: while load balancing is not a requirement in a proxy environment, it does afford a great deal of functionality, scaling and fault tolerance in one. several if not many on this list probably employ them for their proxies and likely other technologies, but they are not all created equal. i recently looked to see if a specific feature was in HAProxy. i was looking to see if HAProxy could reply to a new connection with a RST packet if no pool member was available. the idea behind this is, if all of the proxies are not passing the service check and are marked down by the load balancer, the reply of a RST in the TCP handshake (i.e. SYN - RST, not SYN - SYN/ACK - ACK) tells the browser to failover to the next proxy assigned by the PAC file. where i work, we have this configuration working. the load balancers are configured with the option to send a reset when no proxy is available in the pool. the PAC file assigns all 4 of the proxy VIPs in a specific order based on which proxy VIP is assigned as the primary. In every case, if the primary VIP does not have an available pool member, the browser fails over to the next in the list. failover would happen again, if the secondary VIP replies with a RST during the connection establishing. the process repeats until a TCP connection establishes or all proxies assigned have been exhausted. the browser will use the proxy VIP that it successfully connects to, for the duration of the session. once the browser is closed and reopened, the evaluation of the PAC file occurs again, and the process starts anew. plug-ins such as Proxy Selector are the exception to this, and can be used to reevaluate a PAC file by selecting it for use. we have used this configuration several times, when we found an ISP link was flapping or some other issue more global in nature than just the proxies was affecting our egress and internet access. i can attest to the solution as working and elegantly handling site wide failures. being that the solutions where i work are proprietary commercial products, i wanted to find an open source product that does this. i have been a long time user of HAProxy, and have recommended it for others here, but sadly they cannot perform this function. per their mailing list, they use the network stack of the OS for connection establishment and cannot cause a RST to be sent to the client during a TCP handshake if no pool member is available. they suggested an external helper that manipulates IPTables rules based on a pool member being available. they do not feel that a feature like this belongs in a layer 4/7 reverse proxy application. They are right. HTTP != TCP. i didnt confuse that detail. it was unknown to me that HAProxy could not tie layer 7 status to layer 3/4 actions. the decisions they made and how they architected the app is why they cannot do this, not that it is technically impossible to do it. i may be spoiled because i work with equipment that can do this for me. In particular TCP depends on routers having a full routing map of the entire Internet (provided by BGP) and deciding the best upstream hop based on that global info. Clients have one (and only one) upstream router for each server they want to connect to. i will contest this. my router does not need a full BGP map to route traffic locally on my LAN or remotely out its WAN interface. hell, it does not even run BGP, and i can still get to the intarwebs, no problem. it too, only has one upstream router / default route. In HTTP each proxy (aka router) performs independent upstream connection attempts, failover, and verifies it worked before responding to the client with a final response. Each proxy only has enough detail to check its upstream(s). Each proxy can connect to any server (subject to ACLs). how are you comparing a HTTP proxy (a layer 7 application) to a router (a layer 3 device)? routers route traffic and proxies proxy traffic. very different functions. routers dont look past a certain point in the headers in order to make decisions on where to send the traffic. proxies look all the way to the end of the headers and sometimes into the payload, too. proxies are more akin to a protocol specific firewall. proxies also dont send the incoming traffic out an interface. they terminate the client session, and initiate a new session on behalf of the client. simply because the proxy can elect how to send a request it is
Re: [squid-users] load balancing and site failover
On 26/03/2015 10:26 a.m., Brendan Kearney wrote: On Wed, 2015-03-25 at 15:03 +1300, Amos Jeffries wrote: On 25/03/2015 9:55 a.m., brendan kearney wrote: Was not sure if bugzilla was used for mailing list issues. If you would like me to open one, I will but it looks like the list is working again. Bugzilla is used, list bugs under the project services product. As for your query... On Mar 24, 2015 2:25 PM, Brendan Kearney wrote: On Tue, 2015-03-24 at 10:18 -0400, Brendan Kearney wrote: while load balancing is not a requirement in a proxy environment, it does afford a great deal of functionality, scaling and fault tolerance in one. several if not many on this list probably employ them for their proxies and likely other technologies, but they are not all created equal. i recently looked to see if a specific feature was in HAProxy. i was looking to see if HAProxy could reply to a new connection with a RST packet if no pool member was available. the idea behind this is, if all of the proxies are not passing the service check and are marked down by the load balancer, the reply of a RST in the TCP handshake (i.e. SYN - RST, not SYN - SYN/ACK - ACK) tells the browser to failover to the next proxy assigned by the PAC file. where i work, we have this configuration working. the load balancers are configured with the option to send a reset when no proxy is available in the pool. the PAC file assigns all 4 of the proxy VIPs in a specific order based on which proxy VIP is assigned as the primary. In every case, if the primary VIP does not have an available pool member, the browser fails over to the next in the list. failover would happen again, if the secondary VIP replies with a RST during the connection establishing. the process repeats until a TCP connection establishes or all proxies assigned have been exhausted. the browser will use the proxy VIP that it successfully connects to, for the duration of the session. once the browser is closed and reopened, the evaluation of the PAC file occurs again, and the process starts anew. plug-ins such as Proxy Selector are the exception to this, and can be used to reevaluate a PAC file by selecting it for use. we have used this configuration several times, when we found an ISP link was flapping or some other issue more global in nature than just the proxies was affecting our egress and internet access. i can attest to the solution as working and elegantly handling site wide failures. being that the solutions where i work are proprietary commercial products, i wanted to find an open source product that does this. i have been a long time user of HAProxy, and have recommended it for others here, but sadly they cannot perform this function. per their mailing list, they use the network stack of the OS for connection establishment and cannot cause a RST to be sent to the client during a TCP handshake if no pool member is available. they suggested an external helper that manipulates IPTables rules based on a pool member being available. they do not feel that a feature like this belongs in a layer 4/7 reverse proxy application. They are right. HTTP != TCP. i didnt confuse that detail. it was unknown to me that HAProxy could not tie layer 7 status to layer 3/4 actions. the decisions they made and how they architected the app is why they cannot do this, not that it is technically impossible to do it. i may be spoiled because i work with equipment that can do this for me. In particular TCP depends on routers having a full routing map of the entire Internet (provided by BGP) and deciding the best upstream hop based on that global info. Clients have one (and only one) upstream router for each server they want to connect to. i will contest this. my router does not need a full BGP map to route traffic locally on my LAN or remotely out its WAN interface. hell, it does not even run BGP, and i can still get to the intarwebs, no problem. it too, only has one upstream router / default route. Then your router has more in common with proxy than usual. Its operating with a next-hop packet relay model (OSPF? MPLS?) rather than an end-to-end model (BGP with RIB/FIB). In HTTP each proxy (aka router) performs independent upstream connection attempts, failover, and verifies it worked before responding to the client with a final response. Each proxy only has enough detail to check its upstream(s). Each proxy can connect to any server (subject to ACLs). how are you comparing a HTTP proxy (a layer 7 application) to a router (a layer 3 device)? routers route traffic and proxies proxy traffic. while, routers proxy TCP packets and proxies route HTTP messages. Its the behaviour abstraction I'm talking about here. [if you dont want to dive into theory skip to the end of this mail] The algorithms each are capable of are the same despite differences in details of layer and what designed mechanisms are optimal
Re: [squid-users] load balancing and site failover
On 25/03/2015 9:55 a.m., brendan kearney wrote: Was not sure if bugzilla was used for mailing list issues. If you would like me to open one, I will but it looks like the list is working again. Bugzilla is used, list bugs under the project services product. As for your query... On Mar 24, 2015 2:25 PM, Brendan Kearney wrote: On Tue, 2015-03-24 at 10:18 -0400, Brendan Kearney wrote: while load balancing is not a requirement in a proxy environment, it does afford a great deal of functionality, scaling and fault tolerance in one. several if not many on this list probably employ them for their proxies and likely other technologies, but they are not all created equal. i recently looked to see if a specific feature was in HAProxy. i was looking to see if HAProxy could reply to a new connection with a RST packet if no pool member was available. the idea behind this is, if all of the proxies are not passing the service check and are marked down by the load balancer, the reply of a RST in the TCP handshake (i.e. SYN - RST, not SYN - SYN/ACK - ACK) tells the browser to failover to the next proxy assigned by the PAC file. where i work, we have this configuration working. the load balancers are configured with the option to send a reset when no proxy is available in the pool. the PAC file assigns all 4 of the proxy VIPs in a specific order based on which proxy VIP is assigned as the primary. In every case, if the primary VIP does not have an available pool member, the browser fails over to the next in the list. failover would happen again, if the secondary VIP replies with a RST during the connection establishing. the process repeats until a TCP connection establishes or all proxies assigned have been exhausted. the browser will use the proxy VIP that it successfully connects to, for the duration of the session. once the browser is closed and reopened, the evaluation of the PAC file occurs again, and the process starts anew. plug-ins such as Proxy Selector are the exception to this, and can be used to reevaluate a PAC file by selecting it for use. we have used this configuration several times, when we found an ISP link was flapping or some other issue more global in nature than just the proxies was affecting our egress and internet access. i can attest to the solution as working and elegantly handling site wide failures. being that the solutions where i work are proprietary commercial products, i wanted to find an open source product that does this. i have been a long time user of HAProxy, and have recommended it for others here, but sadly they cannot perform this function. per their mailing list, they use the network stack of the OS for connection establishment and cannot cause a RST to be sent to the client during a TCP handshake if no pool member is available. they suggested an external helper that manipulates IPTables rules based on a pool member being available. they do not feel that a feature like this belongs in a layer 4/7 reverse proxy application. They are right. HTTP != TCP. In particular TCP depends on routers having a full routing map of the entire Internet (provided by BGP) and deciding the best upstream hop based on that global info. Clients have one (and only one) upstream router for each server they want to connect to. In HTTP each proxy (aka router) performs independent upstream connection attempts, failover, and verifies it worked before responding to the client with a final response. Each proxy only has enough detail to check its upstream(s). Each proxy can connect to any server (subject to ACLs). my search for a load balancer solution went through ipvsadm, balance and haproxy before i selected haproxy. haproxy was more feature rich than balance, and easier to implement than ipvsadm. do any other list members have a need for such a feature from their load balancers? do any other list members have site failover solutions that have been tested or used and would consider sharing their design and/or pain points? i am not looking for secret sauce or confidential info, but more high level architecture decisions and such. I havent tested it but this should do what you are asking: acl err http_status 500-505 408 deny_info TCP_RESET err http_reply_access deny err It replaces the response from Squid with a TCP RST packet. Amos ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] load balancing and site failover
Was not sure if bugzilla was used for mailing list issues. If you would like me to open one, I will but it looks like the list is working again. On Mar 24, 2015 2:25 PM, Brendan Kearney bpk...@gmail.com wrote: On Tue, 2015-03-24 at 10:18 -0400, Brendan Kearney wrote: while load balancing is not a requirement in a proxy environment, it does afford a great deal of functionality, scaling and fault tolerance in one. several if not many on this list probably employ them for their proxies and likely other technologies, but they are not all created equal. i recently looked to see if a specific feature was in HAProxy. i was looking to see if HAProxy could reply to a new connection with a RST packet if no pool member was available. the idea behind this is, if all of the proxies are not passing the service check and are marked down by the load balancer, the reply of a RST in the TCP handshake (i.e. SYN - RST, not SYN - SYN/ACK - ACK) tells the browser to failover to the next proxy assigned by the PAC file. where i work, we have this configuration working. the load balancers are configured with the option to send a reset when no proxy is available in the pool. the PAC file assigns all 4 of the proxy VIPs in a specific order based on which proxy VIP is assigned as the primary. In every case, if the primary VIP does not have an available pool member, the browser fails over to the next in the list. failover would happen again, if the secondary VIP replies with a RST during the connection establishing. the process repeats until a TCP connection establishes or all proxies assigned have been exhausted. the browser will use the proxy VIP that it successfully connects to, for the duration of the session. once the browser is closed and reopened, the evaluation of the PAC file occurs again, and the process starts anew. plug-ins such as Proxy Selector are the exception to this, and can be used to reevaluate a PAC file by selecting it for use. we have used this configuration several times, when we found an ISP link was flapping or some other issue more global in nature than just the proxies was affecting our egress and internet access. i can attest to the solution as working and elegantly handling site wide failures. being that the solutions where i work are proprietary commercial products, i wanted to find an open source product that does this. i have been a long time user of HAProxy, and have recommended it for others here, but sadly they cannot perform this function. per their mailing list, they use the network stack of the OS for connection establishment and cannot cause a RST to be sent to the client during a TCP handshake if no pool member is available. they suggested an external helper that manipulates IPTables rules based on a pool member being available. they do not feel that a feature like this belongs in a layer 4/7 reverse proxy application. my search for a load balancer solution went through ipvsadm, balance and haproxy before i selected haproxy. haproxy was more feature rich than balance, and easier to implement than ipvsadm. do any other list members have a need for such a feature from their load balancers? do any other list members have site failover solutions that have been tested or used and would consider sharing their design and/or pain points? i am not looking for secret sauce or confidential info, but more high level architecture decisions and such. trying to send this again, as it was rejected previously. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
[squid-users] load balancing
Good Morning, I have a proxy squid on debian with kerberos authentication and it works fine. I would create a cluster load balancing for 2/3 proxy squid. In particular, the clients connect to the load balancer, that redirects the request to one of the proxies. These proxies will must authenticate through kerberos. Is it possible implement something like that? What can I use? Best regards. Nicola
Re: [squid-users] load balancing
Hi, Yes, it is even pretty easy to accomplish. We are using a dedicated Loadbalancer (but you can of course use LVS as loadbalancer) which is balancing proxy request to 8 squid instances on 4 different real servers with Kerberos authentication. We are not using any cache hierarchy, just 4 standalone squid servers. Just create a virtual loadbalancer IP, configure an DNS-entry for that IP and configure this FQDN (don't use the IP-address because Kerberos won't work) in your client browsers. Create a Kerberos Ticket for this hostname/fqdn (I assume you already did something similiar for your current setup) and use this ticketfile on your squid servers. That's pretty much it. regards Peter On Tue, Nov 8, 2011 at 2:43 PM, Nicola Gentile nikko...@gmail.com wrote: Good Morning, I have a proxy squid on debian with kerberos authentication and it works fine. I would create a cluster load balancing for 2/3 proxy squid. In particular, the clients connect to the load balancer, that redirects the request to one of the proxies. These proxies will must authenticate through kerberos. Is it possible implement something like that? What can I use? Best regards. Nicola
[squid-users] Load balancing WITHOUT parents over multiple WAN connections
*This message was scanned for all current viruses and is certified clean* Hi All, I'm using Squid 3.0. STABLE 8 as my main cache, and I have two other 2.7 caches set up, one on each WAN connection. The only reason we have multiple proxy servers is to load balance; in reality I only need the one 3.0 server, which has access to both WAN links. I don't want any ACLs that force one group of users to one outgoing IP; I just want Squid to see that it has two paths to the internet and use them in a weighted round-robin fashion. I have tried setting up one direct and one parent, but no weighting occurs because Squid prefers direct routes if possible. I have tried to force squid to use an IP address on the machine and set up multiple weighted routes from that IP, but strange things happen with web sites that check source IP address, so it needs to be something that Squid can control. Any suggestions? Thanks, Jon DeLee
Re: [squid-users] Load balancing WITHOUT parents over multiple WAN connections
Jon DeLee wrote: *This message was scanned for all current viruses and is certified clean* Hi All, I'm using Squid 3.0. STABLE 8 as my main cache, and I have two other 2.7 caches set up, one on each WAN connection. The only reason we have multiple proxy servers is to load balance; in reality I only need the one 3.0 server, which has access to both WAN links. I don't want any ACLs that force one group of users to one outgoing IP; I just want Squid to see that it has two paths to the internet and use them in a weighted round-robin fashion. I have tried setting up one direct and one parent, but no weighting occurs because Squid prefers direct routes if possible. I have tried to force squid to use an IP address on the machine and set up multiple weighted routes from that IP, but strange things happen with web sites that check source IP address, so it needs to be something that Squid can control. Any suggestions? For two links, I would think that an external_acl_type that makes a hash of the domain (or perhaps just the top level domain) and returns true for half (or whatever distribution you want) and false for the rest, associated with tcp_outgoing_address would work. external_acl_type domainhash %DST /my/helper/scripts/domainhash acl domainhash external domainhash tcp_outgoing_address 192.168.18.45 domainhash tcp_outgoing_address 10.32.97.43 !domainhash I can't make any promises regarding the fail-over capability of this setup. Thanks, Jon DeLee Chris
[squid-users] Load Balancing Query
Hi All, Want to know if load balancing is possible with squid by maintaining sessions. Health check should be TCP Ports eg: Server A - Active port 8080 Server B - Active port 8080 Client - Squid - Server A and/or B Request 1 comes from 'Client A' Squid forwards the request to 'Server A' Request 2 comes from 'Client A' Squid forwards the request to 'Server A' and so on any further request from 'Client A' squid should only forward to 'Server A' until the session is same if Request 1 comes from 'Client B' Squid forwards the request to 'Server B' Request 2 comes from 'Client B' Squid forwards the request to 'Server B' if 'Server A' fails Squid should forward all the request to 'Server B' //Remy -- Disclaimer and Confidentiality This material has been checked for computer viruses and although none has been found, we cannot guarantee that it is completely free from such problems and do not accept any liability for loss or damage which may be caused. Please therefore check any attachments for viruses before using them on your own equipment. If you do find a computer virus please inform us immediately so that we may take appropriate action. This communication is intended solely for the addressee and is confidential. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. The views expressed in this message are those of the individual sender, and may not necessarily be that of ISA.
Re: [squid-users] Load Balancing Query
Mario Remy Almeida wrote: Hi All, Want to know if load balancing is possible with squid by maintaining sessions. Health check should be TCP Ports eg: Server A - Active port 8080 Server B - Active port 8080 Client - Squid - Server A and/or B Request 1 comes from 'Client A' Squid forwards the request to 'Server A' Request 2 comes from 'Client A' Squid forwards the request to 'Server A' and so on any further request from 'Client A' squid should only forward to 'Server A' until the session is same if Request 1 comes from 'Client B' Squid forwards the request to 'Server B' Request 2 comes from 'Client B' Squid forwards the request to 'Server B' if 'Server A' fails Squid should forward all the request to 'Server B' //Remy HTTP is stateless. It contains no such thing as sessions. That is a browser feature. What you are looking for is something like CARP or sourcehash peering algorithms. They keep all requests for certain URLs sent to the same place (CARP) or all requests for the same IP to the same place (sourcehash). see http://www.squid-cache.org/Doc/config/cache_peer Amos -- Please be using Current Stable Squid 2.7.STABLE6 or 3.0.STABLE15 Current Beta Squid 3.1.0.8 or 3.0.STABLE16-RC1
Re: [squid-users] Load Balancing Query
Hi Amos, Thanks for that, so I need to use carp and sourcehash to do load balancing, right? but where do I specify in squid to monitor the prots? I mean if port 8080 is down on 'ServerA' how Squid will know that it should send the request to 'ServerB' on port 8080? //Remy On Mon, 2009-06-15 at 23:05 +1200, Amos Jeffries wrote: Mario Remy Almeida wrote: Hi All, Want to know if load balancing is possible with squid by maintaining sessions. Health check should be TCP Ports eg: Server A - Active port 8080 Server B - Active port 8080 Client - Squid - Server A and/or B Request 1 comes from 'Client A' Squid forwards the request to 'Server A' Request 2 comes from 'Client A' Squid forwards the request to 'Server A' and so on any further request from 'Client A' squid should only forward to 'Server A' until the session is same if Request 1 comes from 'Client B' Squid forwards the request to 'Server B' Request 2 comes from 'Client B' Squid forwards the request to 'Server B' if 'Server A' fails Squid should forward all the request to 'Server B' //Remy HTTP is stateless. It contains no such thing as sessions. That is a browser feature. What you are looking for is something like CARP or sourcehash peering algorithms. They keep all requests for certain URLs sent to the same place (CARP) or all requests for the same IP to the same place (sourcehash). see http://www.squid-cache.org/Doc/config/cache_peer Amos -- Disclaimer and Confidentiality This material has been checked for computer viruses and although none has been found, we cannot guarantee that it is completely free from such problems and do not accept any liability for loss or damage which may be caused. Please therefore check any attachments for viruses before using them on your own equipment. If you do find a computer virus please inform us immediately so that we may take appropriate action. This communication is intended solely for the addressee and is confidential. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. The views expressed in this message are those of the individual sender, and may not necessarily be that of ISA.
Re: [squid-users] Load Balancing Query
Mario Remy Almeida wrote: Hi Amos, Thanks for that, so I need to use carp and sourcehash to do load balancing, right? only the one you want. but where do I specify in squid to monitor the prots? I mean if port 8080 is down on 'ServerA' how Squid will know that it should send the request to 'ServerB' on port 8080? It's automatic in the background. The latest 2.HEAD and 3.1 have options to configure how long it takes to detect. Other squid attempt ~10 connects and then failover. Amos //Remy On Mon, 2009-06-15 at 23:05 +1200, Amos Jeffries wrote: Mario Remy Almeida wrote: Hi All, Want to know if load balancing is possible with squid by maintaining sessions. Health check should be TCP Ports eg: Server A - Active port 8080 Server B - Active port 8080 Client - Squid - Server A and/or B Request 1 comes from 'Client A' Squid forwards the request to 'Server A' Request 2 comes from 'Client A' Squid forwards the request to 'Server A' and so on any further request from 'Client A' squid should only forward to 'Server A' until the session is same if Request 1 comes from 'Client B' Squid forwards the request to 'Server B' Request 2 comes from 'Client B' Squid forwards the request to 'Server B' if 'Server A' fails Squid should forward all the request to 'Server B' //Remy HTTP is stateless. It contains no such thing as sessions. That is a browser feature. What you are looking for is something like CARP or sourcehash peering algorithms. They keep all requests for certain URLs sent to the same place (CARP) or all requests for the same IP to the same place (sourcehash). see http://www.squid-cache.org/Doc/config/cache_peer Amos -- Please be using Current Stable Squid 2.7.STABLE6 or 3.0.STABLE15 Current Beta Squid 3.1.0.8 or 3.0.STABLE16-RC1
Re: [squid-users] Load Balancing Query
Thanks Amos for the help On Tue, 2009-06-16 at 00:30 +1200, Amos Jeffries wrote: Mario Remy Almeida wrote: Hi Amos, Thanks for that, so I need to use carp and sourcehash to do load balancing, right? only the one you want. but where do I specify in squid to monitor the prots? I mean if port 8080 is down on 'ServerA' how Squid will know that it should send the request to 'ServerB' on port 8080? It's automatic in the background. The latest 2.HEAD and 3.1 have options to configure how long it takes to detect. Other squid attempt ~10 connects and then failover. Amos //Remy On Mon, 2009-06-15 at 23:05 +1200, Amos Jeffries wrote: Mario Remy Almeida wrote: Hi All, Want to know if load balancing is possible with squid by maintaining sessions. Health check should be TCP Ports eg: Server A - Active port 8080 Server B - Active port 8080 Client - Squid - Server A and/or B Request 1 comes from 'Client A' Squid forwards the request to 'Server A' Request 2 comes from 'Client A' Squid forwards the request to 'Server A' and so on any further request from 'Client A' squid should only forward to 'Server A' until the session is same if Request 1 comes from 'Client B' Squid forwards the request to 'Server B' Request 2 comes from 'Client B' Squid forwards the request to 'Server B' if 'Server A' fails Squid should forward all the request to 'Server B' //Remy HTTP is stateless. It contains no such thing as sessions. That is a browser feature. What you are looking for is something like CARP or sourcehash peering algorithms. They keep all requests for certain URLs sent to the same place (CARP) or all requests for the same IP to the same place (sourcehash). see http://www.squid-cache.org/Doc/config/cache_peer Amos -- Disclaimer and Confidentiality This material has been checked for computer viruses and although none has been found, we cannot guarantee that it is completely free from such problems and do not accept any liability for loss or damage which may be caused. Please therefore check any attachments for viruses before using them on your own equipment. If you do find a computer virus please inform us immediately so that we may take appropriate action. This communication is intended solely for the addressee and is confidential. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. The views expressed in this message are those of the individual sender, and may not necessarily be that of ISA.
Re: [squid-users] load balancing
Hi Remy, Just a couple of comments. 1) As per your response, if DNS is down squid is not going to be much happier as it needs that DNS resolution in order to be able to function ;-) 2) WCCP would/could work very nicely for you in a fully transparent configuration. Cost of wccp capable routers plays a role 3) A true load balancer front end like Cisco's content director could also do the job but also runs into cost issues. Methods I've used: 1) Running squid in an LVS (linux virtual server) environment - works but can get fun to configure 2) Add another squid box to the configuration. - Setup this squid so that 10.200.1.2 and 10.200.1.1 are parent caches with CARP enabled - Do not enable any disk storage on this front-end cache This gives you an environment where the parent caches will determine load between them and handle requests as needed. Setting dead_peer_timeout and peer_connect_timeout will also allow relatively quick responses to caches that die. I know this last option is not fully redundant but is a cost effective way of handling the load balancing issue cleanly. Regardt Mario Remy Almeida wrote: Hi All, What I mean to say is.. E.G:- SP 1 = 10.200.2.1 SP 2 = 10.200.2.2 LAN USERS = 10.200.2.x All lan users should connect to SP1 or SP2 depending upon the load and if one of the SP is down the other should take the load. One way of achieving load balance is with DNS proxy1.example.com IN A 10.200.2.1 proxy1.example.com IN A 10.200.2.2 And what if the DNS Server is down and also how to do fail over //Remy On Tue, 2008-12-23 at 09:05 -0600, Luis Daniel Lucio Quiroz wrote: Just remember when using load balancing, if you use digest auth, then you MUST use source persistence. On Tuesday 23 December 2008 08:38:27 Ken Peng wrote: Hi All, any links on how to configure load balancing of squid See the default squid.conf, :)
Re: [squid-users] load balancing
Mario Remy Almeida wrote: Hi All, What I mean to say is.. E.G:- SP 1 = 10.200.2.1 SP 2 = 10.200.2.2 LAN USERS = 10.200.2.x All lan users should connect to SP1 or SP2 depending upon the load and if one of the SP is down the other should take the load. One way of achieving load balance is with DNS proxy1.example.com IN A 10.200.2.1 proxy1.example.com IN A 10.200.2.2 Hi Remy, I agree the DNS server could do the balancing here. But to be more precise DNS is more appropriate in Load-Balancing the other kind of services like SMTP, WEB etc. What I recommend is a router capable of web-traffic redirection like WCCP in Cisco routers. May be if you want to know more about WCCP. This URL http://articles.techrepublic.com.com/5100-10878_11-6175637.html could let you know how it works. Regards, Pritam Regards, Pritam And what if the DNS Server is down and also how to do fail over //Remy On Tue, 2008-12-23 at 09:05 -0600, Luis Daniel Lucio Quiroz wrote: Just remember when using load balancing, if you use digest auth, then you MUST use source persistence. On Tuesday 23 December 2008 08:38:27 Ken Peng wrote: Hi All, any links on how to configure load balancing of squid See the default squid.conf, :) Internal Virus Database is out of date. Checked by AVG - http://www.avg.com Version: 8.0.176 / Virus Database: 270.9.19/1857 - Release Date: 12/19/2008 10:09 AM
Re: [squid-users] load balancing
Hi All, What I mean to say is.. E.G:- SP 1 = 10.200.2.1 SP 2 = 10.200.2.2 LAN USERS = 10.200.2.x All lan users should connect to SP1 or SP2 depending upon the load and if one of the SP is down the other should take the load. One way of achieving load balance is with DNS proxy1.example.com IN A 10.200.2.1 proxy1.example.com IN A 10.200.2.2 And what if the DNS Server is down and also how to do fail over //Remy On Tue, 2008-12-23 at 09:05 -0600, Luis Daniel Lucio Quiroz wrote: Just remember when using load balancing, if you use digest auth, then you MUST use source persistence. On Tuesday 23 December 2008 08:38:27 Ken Peng wrote: Hi All, any links on how to configure load balancing of squid See the default squid.conf, :)
RE: [squid-users] load balancing
Hi All, I was on leave so could not reply. What I mean to say is.. E.G:- SP 1 = 10.200.2.1 SP 2 = 10.200.2.2 LAN USERS = 10.200.2.x All lan users should connect to SP1 or SP2 depending upon the load and if one of the SP is down the other should take the load. One way of achieving load balance is with DNS proxy1.example.com IN A 10.200.2.1 proxy1.example.com IN A 10.200.2.2 And what if the DNS Server is down and also how to do fail over //Remy On Tue, 2008-12-23 at 13:43 +, Mehmet CELIK wrote: Hi, what do you want ? so you mean load-balance.. -- Mehmet CELIK From: malme...@isaaviation.ae To: squid-users@squid-cache.org Date: Tue, 23 Dec 2008 16:21:58 +0400 Subject: [squid-users] load balancing Hi All, any links on how to configure load balancing of squid Regards, Mario __ Windows Live™ ile e-posta kutunuzdaki işlevlerin çok ötesine geçin. Diğer Windows Live™ özelliklerine göz atın.
Re: [squid-users] load balancing
andre wang wrote: hi any details? On Tue, Dec 23, 2008 at 10:38 PM, Ken Peng kenp...@rambler.ru wrote: Hi All, any links on how to configure load balancing of squid See the default squid.conf, :) http://www.suid-cache.org/Doc/config/ Seriously though. 'load balance' means many things. What exactly are you trying to balance? - incoming traffic to a set of web servers through 1+ Squid? - incoming traffic to a set of Squid? - outgoing traffic down a hierarchy? - disk usage within the cache? - DNS traffic through the network? - QoS controls on replies going to clients? - Throttling data peak speeds going to clients? Amos -- Please be using Current Stable Squid 2.7.STABLE5 or 3.0.STABLE11 Current Beta Squid 3.1.0.3
Re: [squid-users] load balancing
Amos Jeffries wrote: andre wang wrote: hi any details? On Tue, Dec 23, 2008 at 10:38 PM, Ken Peng kenp...@rambler.ru wrote: Hi All, any links on how to configure load balancing of squid See the default squid.conf, :) http://www.suid-cache.org/Doc/config/ bah. having a typo-day today: that should be: http://www.squid-cache.org/Doc/config/ Seriously though. 'load balance' means many things. What exactly are you trying to balance? - incoming traffic to a set of web servers through 1+ Squid? - incoming traffic to a set of Squid? - outgoing traffic down a hierarchy? - disk usage within the cache? - DNS traffic through the network? - QoS controls on replies going to clients? - Throttling data peak speeds going to clients? Amos -- Please be using Current Stable Squid 2.7.STABLE5 or 3.0.STABLE11 Current Beta Squid 3.1.0.3
Re: [squid-users] load balancing
If you want to a load balancer for Squid servers then LVS is a good option. Redhat even have a packaged version. J 2008/12/23 Ken Peng kenp...@rambler.ru: Hi All, any links on how to configure load balancing of squid See the default squid.conf, :)
Re: [squid-users] load balancing
hi any details? On Tue, Dec 23, 2008 at 10:38 PM, Ken Peng kenp...@rambler.ru wrote: Hi All, any links on how to configure load balancing of squid See the default squid.conf, :)
[squid-users] load balancing
Hi All, any links on how to configure load balancing of squid Regards, Mario
Re: [squid-users] load balancing
Hi All, any links on how to configure load balancing of squid See the default squid.conf, :)
Re: [squid-users] Load-balancing domain controllers with Squid
On Jan 22, 2008 5:05 PM, Nick Duda [EMAIL PROTECTED] wrote: I've googled around a bit and see things about using the ntlm_auth -b option, but I don't understand how/where? Problem: I have squid setup to do NTLM aunthetication of the proxy users against a local domain controller (samba/winbind installed on the proxy server), pretty straight forward. If the domain controller goes down or gets rebooted, the proxy users start getting popups to authenticate. I typically restart the winbind service when this happens once the DC comes back up. How can I setup squid, or samba/winbind so that when one DC goes down it uses the backup DC? The Samba folks are probably better off in answering this question. I'd be interested in knowing the answer tho. -- /kinkie
[squid-users] Load-balancing domain controllers with Squid
I've googled around a bit and see things about using the ntlm_auth -b option, but I don't understand how/where? Problem: I have squid setup to do NTLM aunthetication of the proxy users against a local domain controller (samba/winbind installed on the proxy server), pretty straight forward. If the domain controller goes down or gets rebooted, the proxy users start getting popups to authenticate. I typically restart the winbind service when this happens once the DC comes back up. How can I setup squid, or samba/winbind so that when one DC goes down it uses the backup DC? - Nick
RE: [squid-users] Load-balancing domain controllers with Squid
Ok cool, I'll post over there. I'm just trying to see if anyone else has done this already with Squidkill 2 birds with one stone here. -Original Message- From: Kinkie [mailto:[EMAIL PROTECTED] Sent: Tuesday, January 22, 2008 11:16 AM To: Nick Duda Cc: squid-users@squid-cache.org Subject: Re: [squid-users] Load-balancing domain controllers with Squid On Jan 22, 2008 5:05 PM, Nick Duda [EMAIL PROTECTED] wrote: I've googled around a bit and see things about using the ntlm_auth -b option, but I don't understand how/where? Problem: I have squid setup to do NTLM aunthetication of the proxy users against a local domain controller (samba/winbind installed on the proxy server), pretty straight forward. If the domain controller goes down or gets rebooted, the proxy users start getting popups to authenticate. I typically restart the winbind service when this happens once the DC comes back up. How can I setup squid, or samba/winbind so that when one DC goes down it uses the backup DC? The Samba folks are probably better off in answering this question. I'd be interested in knowing the answer tho. -- /kinkie
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
On tor, 2007-11-22 at 13:57 +0530, Siju George wrote: method as I use now then even through the outgoing address will be changed for 50% of the packets those same packets will be routed out through the default interface only :-( There is two related components needed to make multipath routing work properly for hosts connected to multiple ISPs a) You need multi-path balancing support in the routing, balancing the outgoing sessions on the available paths. The primary function of this is to select the proper source IP. b) You need policy routing based on source IP to route already assigned traffic out via the correct ISP. Regards Henrik signature.asc Description: This is a digitally signed message part
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
On tor, 2007-11-22 at 08:19 +0100, Matus UHLAR - fantomas wrote: We have squid servers behind L3 switch (balancer) which uses the 'sourcehash' technique for balancing requests because of this reason. Maybe such functionality could be integrated into squid. Or into packetfilter :) That's a good coding project for some interested volunteer. Extend tcp_outgoing_address with a sourcehash selector.. Regards Henrik signature.asc Description: This is a digitally signed message part
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
On Nov 22, 2007 8:33 AM, Ming-Ching Tiew [EMAIL PROTECTED] wrote: From: Siju George [EMAIL PROTECTED] But not something I would recommend. Many sites dislikes clients coming from more than one IP during the same session. The client IP is often embedded in session cookies etc, making the session fail if the IP changes. Yes Henrik. Such sites are identified and there is af firewall rule added to PF in OpenBSD to route them through the same interface.. But it is not a problem with majority of the sites. Perhaps it will be interesting for squid to have an acl called random :- ( is there one already ? ) eg acl rnd random 50 # 50 % tcp_outgoing_address x.x.x.x rnd --- use x 50 % of time tcp_outgoing_address y.y.y.y rnd use y 50% of time tcp_outgoing_address z.z.z.z --- have to provide a default in case nothing is matched And the random acl can be used together with other acl too ! eg acl link1 dst . tcp_outgoing_address x.x.x.x link1 rnd :-) If you use the http://www.openbsd.org/faq/pf/pools.html#outgoing method as I use now then even through the outgoing address will be changed for 50% of the packets those same packets will be routed out through the default interface only :-( Thankyou so much kind Regards Siju
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
There is another replacement for load-balancing, known as ClarkConnect. It is a perfect load balancer, firewall, with many other different solutions. Try it... http://www.clarkconnect.com/ -- Thank you, Javed - Original Message From: Siju George [EMAIL PROTECTED] To: Ming-Ching Tiew [EMAIL PROTECTED] Cc: squid-users squid-users@squid-cache.org Sent: Thursday, November 22, 2007 1:27:11 PM Subject: Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections On Nov 22, 2007 8:33 AM, Ming-Ching Tiew [EMAIL PROTECTED] wrote: From: Siju George [EMAIL PROTECTED] But not something I would recommend. Many sites dislikes clients coming from more than one IP during the same session. The client IP is often embedded in session cookies etc, making the session fail if the IP changes. Yes Henrik. Such sites are identified and there is af firewall rule added to PF in OpenBSD to route them through the same interface.. But it is not a problem with majority of the sites. Perhaps it will be interesting for squid to have an acl called random :- ( is there one already ? ) eg acl rnd random 50 # 50 % tcp_outgoing_address x.x.x.x rnd --- use x 50 % of time tcp_outgoing_address y.y.y.y rnd use y 50% of time tcp_outgoing_address z.z.z.z --- have to provide a default in case nothing is matched And the random acl can be used together with other acl too ! eg acl link1 dst . tcp_outgoing_address x.x.x.x link1 rnd :-) If you use the http://www.openbsd.org/faq/pf/pools.html#outgoing method as I use now then even through the outgoing address will be changed for 50% of the packets those same packets will be routed out through the default interface only :-( Thankyou so much kind Regards Siju Be a better pen pal. Text or chat with friends inside Yahoo! Mail. See how. http://overview.mail.yahoo.com/
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
If you use the http://www.openbsd.org/faq/pf/pools.html#outgoing method as I use now then even through the outgoing address will be changed for 50% of the packets those same packets will be routed out through the default interface only :-( First of all I don't know anything about OpenBSD but we have to know, making squid use a particular tcp_outgoing_address is as much as what we can ask squid to do, actual routing decision should rightfully be left to the OPERATING SYSTEM. In Linux, all these things are addressed :- 1. You can use POLICY ROUTING - one of the capabilities of policy routing is be able to route based on SOURCE IP instead of destination IP. 2. You can use MULTI PATH ROUTING - ie you can start a session based on certain relative weight you assign to a default route. 3. You can also use netfilter 'recent'/CONNMARK match - they allow you to tie a session to an interface for a configurable amount of time. I might not be even exhaustive here. All in all, my conclusion is this :- You really have to look hard to the operating system for this instead of squid. If you have exhausted the capabilities of OpenBSD, you would have to throw it away ! :-)
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
On Nov 21, 2007 12:21 PM, Tek Bahadur Limbu [EMAIL PROTECTED] wrote: Hi George, Siju George wrote: Hi, I have a System with two Internet connections. Is it possible to configure squid to load balance out going internet traffic through those two Internet Connections? To keep things simple, you can just use the tcp_outgoing_address parameter in squid.conf. It didn't work :-( I am running OpenSBD and using the route-to option in pf.conf http://www.openbsd.org/faq/pf/pools.html#outgoing to load balance Internet connections. It is not multipath routing with two default routes. Thank you so much for the response :-) Kind Regards Siju
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
On Nov 22, 2007 2:16 AM, Henrik Nordstrom [EMAIL PROTECTED] wrote: On ons, 2007-11-21 at 13:50 +0900, Adrian Chadd wrote: On Wed, Nov 21, 2007, Ming-Ching Tiew wrote: As far as I know, you could do split access using the 'tcp_outgoing_address' method, but you can't get squid to use it in round-robin manner. I might be wrong. :-) I don't think there is, but making squid do that with a small source patch wouldn't be difficult. But not something I would recommend. Many sites dislikes clients coming from more than one IP during the same session. The client IP is often embedded in session cookies etc, making the session fail if the IP changes. Yes Henrik. Such sites are identified and there is af firewall rule added to PF in OpenBSD to route them through the same interface.. But it is not a problem with majority of the sites. Thank you so much for the response :-) Kind Regards Siju
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
From: Siju George [EMAIL PROTECTED] But not something I would recommend. Many sites dislikes clients coming from more than one IP during the same session. The client IP is often embedded in session cookies etc, making the session fail if the IP changes. Yes Henrik. Such sites are identified and there is af firewall rule added to PF in OpenBSD to route them through the same interface.. But it is not a problem with majority of the sites. Perhaps it will be interesting for squid to have an acl called random :- ( is there one already ? ) eg acl rnd random 50 # 50 % tcp_outgoing_address x.x.x.x rnd --- use x 50 % of time tcp_outgoing_address y.y.y.y rnd use y 50% of time tcp_outgoing_address z.z.z.z --- have to provide a default in case nothing is matched And the random acl can be used together with other acl too ! eg acl link1 dst . tcp_outgoing_address x.x.x.x link1 rnd :-)
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
It sounds like a coding project - are you volunteering? :) Adrian On Thu, Nov 22, 2007, Ming-Ching Tiew wrote: From: Siju George [EMAIL PROTECTED] But not something I would recommend. Many sites dislikes clients coming from more than one IP during the same session. The client IP is often embedded in session cookies etc, making the session fail if the IP changes. Yes Henrik. Such sites are identified and there is af firewall rule added to PF in OpenBSD to route them through the same interface.. But it is not a problem with majority of the sites. Perhaps it will be interesting for squid to have an acl called random :- ( is there one already ? ) eg acl rnd random 50 # 50 % tcp_outgoing_address x.x.x.x rnd --- use x 50 % of time tcp_outgoing_address y.y.y.y rnd use y 50% of time tcp_outgoing_address z.z.z.z --- have to provide a default in case nothing is matched And the random acl can be used together with other acl too ! eg acl link1 dst . tcp_outgoing_address x.x.x.x link1 rnd :-) -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
Hi George, Siju George wrote: On Nov 21, 2007 12:21 PM, Tek Bahadur Limbu [EMAIL PROTECTED] wrote: Hi George, Siju George wrote: Hi, I have a System with two Internet connections. Is it possible to configure squid to load balance out going internet traffic through those two Internet Connections? To keep things simple, you can just use the tcp_outgoing_address parameter in squid.conf. It didn't work :-( I am running OpenSBD and using the route-to option in pf.conf http://www.openbsd.org/faq/pf/pools.html#outgoing I am not familiar with OpenBSD systems. How many network interface cards does this OpenBSD system have? And how is your network setup layout? to load balance Internet connections. It is not multipath routing with two default routes. Why don't you create 2 alias in your network interface pointing to the 2 routers having internet connectivity and then use the tcp_outgoing_address parameter? Thanking you... Thank you so much for the response :-) Kind Regards Siju -- With best regards and good wishes, Yours sincerely, Tek Bahadur Limbu System Administrator (TAG/TDG Group) Jwl Systems Department Worldlink Communications Pvt. Ltd. Jawalakhel, Nepal http://www.wlink.com.np http://teklimbu.wordpress.com
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
On Wed, Nov 21, 2007, Ming-Ching Tiew wrote: As far as I know, you could do split access using the 'tcp_outgoing_address' method, but you can't get squid to use it in round-robin manner. I might be wrong. :-) On ons, 2007-11-21 at 13:50 +0900, Adrian Chadd wrote: I don't think there is, but making squid do that with a small source patch wouldn't be difficult. On 21.11.07 21:46, Henrik Nordstrom wrote: But not something I would recommend. Many sites dislikes clients coming from more than one IP during the same session. The client IP is often embedded in session cookies etc, making the session fail if the IP changes. We have squid servers behind L3 switch (balancer) which uses the 'sourcehash' technique for balancing requests because of this reason. Maybe such functionality could be integrated into squid. Or into packetfilter :) -- Matus UHLAR - fantomas, [EMAIL PROTECTED] ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. There's a long-standing bug relating to the x86 architecture that allows you to install Windows. -- Matthew D. Fuller
[squid-users] load balancing traffic through squid on systems with 2 Internet connections
Hi, I have a System with two Internet connections. Is it possible to configure squid to load balance out going internet traffic through those two Internet Connections? Thank you so much Kind Regards Siju
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
From: Siju George [EMAIL PROTECTED] I have a System with two Internet connections. Is it possible to configure squid to load balance out going internet traffic through those two Internet Connections? This is assuming that you are running Linux :- Just set up multiple routing and weight assignment. You might have to turn off kernel option which caches multiple routing.
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
From: Ming-Ching Tiew [EMAIL PROTECTED] This is assuming that you are running Linux :- Just set up multiple routing and weight assignment. You might have to turn off kernel option which caches multiple routing. I meant MULTI PATH routing. Important Warning! *** This electronic communication (including any attached files) may contain confidential and/or legally privileged information and is only intended for the use of the person to whom it is addressed. If you are not the intended recipient, you do not have permission to read, use, disseminate, distribute, copy or retain any part of this communication or its attachments in any form. If this e-mail was sent to you by mistake, please take the time to notify the sender so that they can identify the problem and avoid any more mistakes in sending e-mail to you. The unauthorised use of information contained in this communication or its attachments may result in legal action against any person who uses it.
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
On Nov 21, 2007 6:29 AM, Ming-Ching Tiew [EMAIL PROTECTED] wrote: From: Ming-Ching Tiew [EMAIL PROTECTED] This is assuming that you are running Linux :- Just set up multiple routing and weight assignment. You might have to turn off kernel option which caches multiple routing. Thank you so much Ching for your kind response:-) I am running OpenBSD with route-to option in PF http://www.openbsd.org/faq/pf/pools.html#outgoing Is there any option to do it in the squid.conf file? I know there is a tcp_outgoing_address option. just wondering if it is possible to make it use all outgoing IP address in a round-robin manner :-) Thank you so much once again. Kind Regards Siju
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
From: Siju George [EMAIL PROTECTED] Is there any option to do it in the squid.conf file? I know there is a tcp_outgoing_address option. just wondering if it is possible to make it use all outgoing IP address in a round-robin manner :-) As far as I know, you could do split access using the 'tcp_outgoing_address' method, but you can't get squid to use it in round-robin manner. I might be wrong. :-)
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
On Wed, Nov 21, 2007, Ming-Ching Tiew wrote: As far as I know, you could do split access using the 'tcp_outgoing_address' method, but you can't get squid to use it in round-robin manner. I might be wrong. :-) I don't think there is, but making squid do that with a small source patch wouldn't be difficult. Adrian -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
Re: [squid-users] load balancing traffic through squid on systems with 2 Internet connections
Hi George, Siju George wrote: Hi, I have a System with two Internet connections. Is it possible to configure squid to load balance out going internet traffic through those two Internet Connections? To keep things simple, you can just use the tcp_outgoing_address parameter in squid.conf. Thanking you... Thank you so much Kind Regards Siju -- With best regards and good wishes, Yours sincerely, Tek Bahadur Limbu System Administrator (TAG/TDG Group) Jwl Systems Department Worldlink Communications Pvt. Ltd. Jawalakhel, Nepal http://www.wlink.com.np http://teklimbu.wordpress.com
[squid-users] Load Balancing Problem
Hi, I need help for load balancing my Squid with two ADSL links. I have try to run additional two Squid parent instances on the same machine but with different IPs binded by using tcp_outgoing_address. It works, but sometimes it generate some error on main Squid like this and make the speed going slower than using just one Squid with single ADSL link. 2007/09/19 23:59:16| TCP connection to 192.168.1.254/3128 failed 2007/09/19 23:59:38| TCP connection to 192.168.2.254/3128 failed 2007/09/19 23:59:39| TCP connection to 192.168.2.254/3128 failed 2007/09/20 00:00:21| TCP connection to 192.168.2.254/3128 failed 2007/09/20 00:00:28| TCP connection to 192.168.2.254/3128 failed 2007/09/20 00:00:30| TCP connection to 192.168.2.254/3128 failed 2007/09/20 00:00:31| TCP connection to 192.168.2.254/3128 failed 2007/09/20 00:00:33| TCP connection to 192.168.1.254/3128 failed 2007/09/20 00:00:41| TCP connection to 192.168.1.254/3128 failed 2007/09/20 00:00:42| TCP connection to 192.168.1.254/3128 failed Here is my topology. Main -(round-robin) Parent1/Parent2 -(tcp_outgoing_address) ISP1/ISP2 Any idea about this error ? Best regards, -- Lazuardi Nasution Laboratorium Sistem Kendali Komputer LABTEK 8 Lantai 2 ITB Ganesha 10 Bandung 40132 +628122142597 +622291230584
Re: [squid-users] Load balancing algorithms for an accelerator
On 5/9/07, Henrik Nordstrom [EMAIL PROTECTED] wrote: Is there any way to balance based on least connections, or something else? Not today, but probably quite easy to add. How would I go about getting this on a developer's radar screen? I don't think this is something I could do myself. Thanks, Sean -- Sean Walberg [EMAIL PROTECTED]http://ertw.com/
Re: [squid-users] Load balancing algorithms for an accelerator
On Fri, May 11, 2007, Sean Walberg wrote: On 5/9/07, Henrik Nordstrom [EMAIL PROTECTED] wrote: Is there any way to balance based on least connections, or something else? Not today, but probably quite easy to add. How would I go about getting this on a developer's radar screen? I don't think this is something I could do myself. You can submit a Wishlist request. I can add it to the Wiki. You can attach a bounty, or you can say you'll donate to the Squid project on completion. Adrian
Re: [squid-users] Load balancing algorithms for an accelerator
You can setup a IPVS load balancer in front of your squid pool. I use it load balance my 10 squid servers. See http://www.linuxvirtualserver.org/ mike At 07:10 AM 5/11/2007, Adrian Chadd wrote: On Fri, May 11, 2007, Sean Walberg wrote: On 5/9/07, Henrik Nordstrom [EMAIL PROTECTED] wrote: Is there any way to balance based on least connections, or something else? Not today, but probably quite easy to add. How would I go about getting this on a developer's radar screen? I don't think this is something I could do myself. You can submit a Wishlist request. I can add it to the Wiki. You can attach a bounty, or you can say you'll donate to the Squid project on completion. Adrian
[squid-users] Load balancing algorithms for an accelerator
Hello, I am successfully using Squid 2.6STABLE12 as a reverse proxy to several backend servers. Each backend web server is defined similar to this: cache_peer x.x.x.x parent 80 0 no-query originserver monitorurl=/squidcheck.php monitorinterval=15 round-robin login=PASS name=webX no-digest With round-robin load balancing sometimes a peer gets busier than others and takes longer to service queries, but Squid continues to give it the same number of connections. For instance right now I'm checking the Peer Cache Statistics and my connections are at 20/19/30/74. Browsing through squid.conf.default it would appear my only other option is sourcehash which would suffer from the same problems. Is there any way to balance based on least connections, or something else? Thanks, Sean -- Sean Walberg [EMAIL PROTECTED]http://ertw.com/
Re: [squid-users] Load balancing algorithms for an accelerator
ons 2007-05-09 klockan 08:13 -0500 skrev Sean Walberg: Is there any way to balance based on least connections, or something else? Not today, but probably quite easy to add. Regards Henrik signature.asc Description: Detta är en digitalt signerad meddelandedel
[squid-users] Load Balancing in ICAP - reg
Hi All, I am just checking the Load balancing feature available in squid-icap setup. I had defined the icap related acls like below and ICAP server is running is both the machines described icap_enable on icap_service service_1 respmod_precache 0 icap://172.16.1.35:1344/respmod icap_service service_1 respmod_precache 0 icap://172.16.1.53:1344/respmod icap_class class_1 service_1 icap_access class_1 allow all In this setup, the requests are always handled by the first machine defined in service_1, in this case, 172.16.1.35. When 172.16.1.35 is down, then the requests are handled by 172.16.1.53. When the load is very high, will both the machines will balance the load? Thanks, Selvi
Re: [squid-users] Load Balancing in ICAP - reg
tis 2007-02-13 klockan 17:15 +0530 skrev Selvi: Hi All, I am just checking the Load balancing feature available in squid-icap setup. Squid-2.x patched with an ICAP patch, or Squid-3 with ICAP enabled? Regards Henrik signature.asc Description: Detta är en digitalt signerad meddelandedel
Re: [squid-users] Load Balancing in ICAP - reg
Hi, Currently i am using Squid-2.x patched with ICAP. Thanks, Selvi - Original Message - From: Henrik Nordstrom [EMAIL PROTECTED] To: Selvi [EMAIL PROTECTED] Cc: squid-users@squid-cache.org Sent: Wednesday, February 14, 2007 3:57 AM Subject: Re: [squid-users] Load Balancing in ICAP - reg
[squid-users] Load Balancing Squids
Hey Guys, I have a situation here. I have a load balancer sending the requests to two differents squids. I tryied to use cache_peer with proxy only option, but I get TCP Denied. What should I do on both confs? Add both as parents with proxy only option or what? Thanxs in advance, JOC
Re: [squid-users] Load Balancing Squids
I've got the same kind of thing going and I have it working with sibling-mode (not parent). Might want to check your firewall stuff and make sure that each squid is listening on 3130 for peer requests (forget what the option is). My problem is what happens when the real server behind the proxies dies? How does squid handle a failure like that? - Steve On Thu, 28 Sep 2006, Jose Octavio de Castro Neves Jr wrote: Date: Thu, 28 Sep 2006 19:00:52 -0300 From: Jose Octavio de Castro Neves Jr [EMAIL PROTECTED] To: squid-users@squid-cache.org Subject: [squid-users] Load Balancing Squids Hey Guys, I have a situation here. I have a load balancer sending the requests to two differents squids. I tryied to use cache_peer with proxy only option, but I get TCP Denied. What should I do on both confs? Add both as parents with proxy only option or what? Thanxs in advance, JOC -- Steve Webb - Sr. Linux System Administrator Email: [EMAIL PROTECTED] Cell: 303-564-4269, Office: 303-497-9368 YIM: scumola
[squid-users] Load balancing squid with squid
I'm thinking to load balancing 2 squid processes with 1 squid. Is this idea stupid? Landscape: 1 SMP machine with: squid1 with cache_dir (a real piece of disk) bind localhost squid2 with cache_dir (a real piece of disk) bind localhost squidv with cache_dir null cache_peer 127.0.0.1 parent 3128 3130 no-query proxy-only round-robin cache_peer 127.0.0.1 parent 4128 4130 no-query proxy-only round-robin never_direct allow all Can someone supply me any comments? Bye Stefano
Re: [squid-users] Load balancing squid with squid
On Friday 18 November 2005 14:31, Stefano Mason wrote: I'm thinking to load balancing 2 squid processes with 1 squid. Is this idea stupid? Landscape: 1 SMP machine with: squid1 with cache_dir (a real piece of disk) bind localhost squid2 with cache_dir (a real piece of disk) bind localhost squidv with cache_dir null cache_peer 127.0.0.1 parent 3128 3130 no-query proxy-only round-robin cache_peer 127.0.0.1 parent 4128 4130 no-query proxy-only round-robin never_direct allow all Can someone supply me any comments? Although this would technically work I don't see any gain here - just more overhead. IMHO two Squids on the same server won't make things go faster than a single Squid. If your proxies are not fast enough (CPU, network bandwidth, etc.) then consider setting up Squids on different servers and put a decent hardware load balancer in front of it. We use a Foundry load balancer and four Squid nodes here. Btw, you can use multiple cache directories (cache_dir) in case you want to use different hard disks to speed up the disk cache. Christoph -- ~ ~ .signature [Modified] 2 lines --100%--2,41 All
Re: [squid-users] Load Balancing
what about caching in your setup? this info will surely help you to design your setup. I am using one proxy server and two caching proxy servers and the total number of users in my setup are much more than yours. I would like to ask you one question here as you have done with transparent proxy. How are you dealing with https(port 443) request from your transparent proxy setup? -- Sushil. On Sat, 22 Oct 2005, Paras pradhan wrote: hi: Currently i have a network running squid as a transparent proxy server serving around 500 users. Around 40% of the users ie ( 200 users) use the network and the performance is good with very very little problem.. now my network is about to increse and almost 1200 Users will be using the proxy server. Presenly i am running a PIII Xeon processor Server with SCSI Hard Drives and 1GB of RAM. Now i need some suggestions for 1200 users. Do i need to: Upgrade or Purchase a new server OR use load balancing using 3 or 4 another servers OR just tune the currnet server for the best performance. What is the best option for my case assuming 50% of my 1200 users(ie 600) will be using the internet at the same time.. Thanks Paras.
[squid-users] Load Balancing
hi: Currently i have a network running squid as a transparent proxy server serving around 500 users. Around 40% of the users ie ( 200 users) use the network and the performance is good with very very little problem.. now my network is about to increse and almost 1200 Users will be using the proxy server. Presenly i am running a PIII Xeon processor Server with SCSI Hard Drives and 1GB of RAM. Now i need some suggestions for 1200 users. Do i need to: Upgrade or Purchase a new server OR use load balancing using 3 or 4 another servers OR just tune the currnet server for the best performance. What is the best option for my case assuming 50% of my 1200 users(ie 600) will be using the internet at the same time.. Thanks Paras.
[squid-users] Load balancing redirector problem with virtual hosts
Hi all -
Okay, I've poked around a bunch looking for possible solutions to this
problem but can't seem to make any headway.
I'm implementing a simple load balancing solution between two origin servers
one of which is running squid and must also handle a number of
non-accelerated virtual hosts.
The set up looks like this:
Primary server:
Squid in surrogate mode listening on 80, with redirector for all URIs
Apache, bound to 127.0.0.1 listening on 80
Serving a number of name-based virtual hosts
Secondary server:
Apache bound to IP listening on 80
I have squid running successfully for all virtual hosts using the single
primary origin server with the following directives:
httpd_accel_host 127.0.0.1
httpd_accel_port 80
httpd_accel_single_host on
httpd_accel_uses_host_header on
If I try to add support for my simple redirector (listed below) using the
following directives, I can no longer connect to any of the virtual hosts.
Client connections end up timing out (WARNING: Closing client 24.10.x.x
connection due to lifetime timeout).
redirect_program /path/to/load_balancer.pl
redirect_rewrites_host_header off
httpd_accel_host 127.0.0.1
httpd_accel_port 80
httpd_accel_single_host off
httpd_accel_uses_host_header on
The redirector (pared down for simplicity; eventually will direct to
secondary server too):
#!/usr/bin/perl -w
$|++;
while () {
print http://127.0.0.1;;
}
I'm sure this is a matter of balancing the different confg directives but I
haven't seen an config exactly like this before.
Any insight would be greatly appreciated!
Todd
[squid-users] Load balancing between 3 squid
Hi guys I'm using squid as cache server , How i can load balance the http traffic between this 3 cache server with round robin algorithm, Supposing that i,ve route http request from cisco router to these cache server,any solution please , I've implement some solution ,such as load balancer on linux,using some software from surceforge,but is not very stable, Can you give some solution, Regards Abbas Salehi
Re: [squid-users] Load balancing between 3 squid
On 07.07 00:41, Abbas Salehi wrote: I'm using squid as cache server , How i can load balance the http traffic between this 3 cache server with round robin algorithm, do you use 1 or 3 cache servers? Supposing that i,ve route http request from cisco router to these cache server,any solution please , just as normal, unless you want to use SLB... then you can look at this thread: http://www.squid-cache.org/mail-archive/squid-users/200506/0662.html I've implement some solution ,such as load balancer on linux,using some software from surceforge,but is not very stable, -- Matus UHLAR - fantomas, [EMAIL PROTECTED] ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. He who laughs last thinks slowest.
Re: [squid-users] Load Balancing
On Sun, 27 Feb 2005, Dominik Jais wrote: delay_parameters 1 -1/-1 13000/13000 I'm using the configuration above. My problem is, that eath user gots 13kbps, but the 13000 mean bytes = 104kbps Is this a documentation bug? The above setting will give each user ca 104kbit/s or 12.5KByte/s. Most clients displays download rate in KByte/s, not kbit/s. Regards Henrik
[squid-users] Load Balancing
As it is described in the Squid Docomentation: # A pair of delay parameters is written restore/maximum, where restore is # the number of bytes (not bits - modem and network speeds are usually # quoted in bits) per second placed into the bucket, and maximum is the # maximum number of bytes which can be in the bucket at any time. # # For example, if delay pool number 1 is a class 2 delay pool as in the # above example, and is being used to strictly limit each host to 64kbps # (plus overheads), with no overall limit, the line is: # delay_parameters 1 -1/-1 13000/13000 I'm using the configuration above. My problem is, that eath user gots 13kbps, but the 13000 mean bytes = 104kbps Is this a documentation bug? greetings Dominik Jais
Re: [squid-users] Load balancing query
On Fri, 28 Jan 2005, Carinus Carelse wrote: I have two external networks connected and I have one proxy server that is servicing about 500 users. I would like the squid box to do some load balancing across the two external link so that the http traffic does not just go through the one link. Is this possible and how do i go about setting this up. This is best done by route load balancing in your OS. But you can use tcp_outgoing_address to make Squid select which link to use for certain kinds of request. Regards Henrik
[squid-users] Load balancing query
I have two external networks connected and I have one proxy server that is servicing about 500 users. I would like the squid box to do some load balancing across the two external link so that the http traffic does not just go through the one link. Is this possible and how do i go about setting this up. Carinus
[squid-users] load balancing
greetings I have a new squid server i want to put in parallel with my existing system. i read most of the Docs and Faq's but still have not come up with a good understanding for my scenario. I'm running a transparent cache with squid guard. My problem seems to be How do I split my traffic ? or how do i create a failover if one squid server becomes to busy? here is my topology --[ L3 def route]---[ squid1]-- ---[squid2]-- right now i my problem is that I have only one default route option coming from my layer 3 device. therefore I cannot split my subnets and force 1/2 to one interface and the other half to another. I do have multiple cards in each squid box. DNS round robin won't work in this setup because i am forcing all unknown traffic to squid1. (correct me if I'm wrong) any advice? --- jeff donovan basd network operations (610) 807 5571 x41 AIM xtdonovan
Re: [squid-users] load balancing
On Mon, 2 Aug 2004, Jeff Donovan wrote: I'm running a transparent cache with squid guard. My problem seems to be How do I split my traffic? This you do in your router in case of transparently intercepting proxies.. If you are not using a TCP interception device capable of tracking individual connections then this is most easily done by splitting the destination IP address space among the caches. CARP does so automatically for you. or how do i create a failover if one squid server becomes to busy? best done by an external load balancer. Linux Virtual Server is a good free one, highend routers switches usually also have reasonable load balancing functions built in. Regards Henrik
Re: [squid-users] load balancing
On Aug 2, 2004, at 9:25 AM, Henrik Nordstrom wrote: On Mon, 2 Aug 2004, Jeff Donovan wrote: I'm running a transparent cache with squid guard. My problem seems to be How do I split my traffic? This you do in your router in case of transparently intercepting proxies.. If you are not using a TCP interception device capable of tracking individual connections then this is most easily done by splitting the destination IP address space among the caches. CARP does so automatically for you. or how do i create a failover if one squid server becomes to busy? best done by an external load balancer. Linux Virtual Server is a good free one, highend routers switches usually also have reasonable load balancing functions built in. would this be a placed on a linux server in front of the two cache's or does it run on the same device as squid? Unfortunately my layer 3 switch does not allow for multiple default routes or even a policy route. Otherwise i would have just routed my traffic to the new interface. -j --- jeff donovan basd network operations (610) 807 5571 x41 AIM xtdonovan
Re: [squid-users] load balancing
On Mon, 2 Aug 2004, Jeff Donovan wrote: would this be a placed on a linux server in front of the two cache's or does it run on the same device as squid? Both possibilities existsm Regards Henrik
Re: [squid-users] load balancing
LVS can be placed on the same server, but it is good if you a new Linux Box for it, I am not sure but last time I used it, I need to recompile the kernel. Please do check with the latest Docs. And why can not you do the Roundrobin. It is always a good option. And yeah most convinient, you can use a PAC file. --- Jeff Donovan [EMAIL PROTECTED] wrote: On Aug 2, 2004, at 9:25 AM, Henrik Nordstrom wrote: On Mon, 2 Aug 2004, Jeff Donovan wrote: I'm running a transparent cache with squid guard. My problem seems to be How do I split my traffic? This you do in your router in case of transparently intercepting proxies.. If you are not using a TCP interception device capable of tracking individual connections then this is most easily done by splitting the destination IP address space among the caches. CARP does so automatically for you. or how do i create a failover if one squid server becomes to busy? best done by an external load balancer. Linux Virtual Server is a good free one, highend routers switches usually also have reasonable load balancing functions built in. would this be a placed on a linux server in front of the two cache's or does it run on the same device as squid? Unfortunately my layer 3 switch does not allow for multiple default routes or even a policy route. Otherwise i would have just routed my traffic to the new interface. -j --- jeff donovan basd network operations (610) 807 5571 x41 AIM xtdonovan = Regards, Mohsin Khan CCNA ( Cisco Certified Network Associate 2.0 ) http://pk.aaghaz.net Happy is the one who can smile __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
[squid-users] load balancing
Hi All Please could I have either some ideas or solutions to the following: I work for a BIG company, providing internet services for many customers. I am planning to install a load balancing plan for our cache server, which will obviously load balance, but will also have a redundant feature. What can I do this with? Basically have to servers service the caching, but they must look like one server ... I am sure I sound stupid, but just to make sure. Regards Gert Brits
[squid-users] RE : [squid-users] load balancing
-Message d'origine- De : Gert Brits [mailto:[EMAIL PROTECTED] Envoyé : jeudi 22 juillet 2004 13:40 À : [EMAIL PROTECTED] Org Objet : [squid-users] load balancing Hi All Please could I have either some ideas or solutions to the following: I work for a BIG company, providing internet services for many customers. I am planning to install a load balancing plan for our cache server, which will obviously load balance, but will also have a redundant feature. What can I do this with? You may perform using either solution like LVS (Linux Virtual Server) http://www.linuxvirtualserver.org or load-balancing dedicated equipment such as Radware Cache Server Director (or Web Server Director), F5 BIG IP, Nortel Alteon, Cisco (I wouldn't recommend since they are not good in this market, really). The thing is you have an equipment (either special switchs or linux servers) that dispatches the incoming requests to real servers. The load-balancer know the states of each real servers to prevent dispatching requests to an overloaded server. And to avoir single point of failure, the dispatcher is doubled in an active/passive way. Basically have to servers service the caching, but they must look like one server ... I am sure I sound stupid, but just to make sure. I hope it will help you. Sincerely yours, Mathieu.
RE: [squid-users] load balancing
Do a search for round robin dns multiple a records clustering redundancy failover. That should give good coverage on the techniques Also...visiting www.squid-cache.org and searching for failover will help. Chris Perreault -Original Message- From: Gert Brits [mailto:[EMAIL PROTECTED] Sent: Thursday, July 22, 2004 7:40 AM To: [EMAIL PROTECTED] Org Subject: [squid-users] load balancing Hi All Please could I have either some ideas or solutions to the following: I work for a BIG company, providing internet services for many customers. I am planning to install a load balancing plan for our cache server, which will obviously load balance, but will also have a redundant feature. What can I do this with? Basically have to servers service the caching, but they must look like one server ... I am sure I sound stupid, but just to make sure. Regards Gert Brits
RE: [squid-users] RE : [squid-users] load balancing
Awesome help thanks I knew about the hardware way ... let me investigate the Linux way ;-) Regards Gert Brits Senior Engineer Technology Concepts Tel +27 11 803 2169 Fax +27 11 803 2189 http://www.techconcepts.co.za -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Thursday, July 22, 2004 1:55 PM To: [EMAIL PROTECTED] Subject: [squid-users] RE : [squid-users] load balancing -Message d'origine- De : Gert Brits [mailto:[EMAIL PROTECTED] Envoyé : jeudi 22 juillet 2004 13:40 À : [EMAIL PROTECTED] Org Objet : [squid-users] load balancing Hi All Please could I have either some ideas or solutions to the following: I work for a BIG company, providing internet services for many customers. I am planning to install a load balancing plan for our cache server, which will obviously load balance, but will also have a redundant feature. What can I do this with? You may perform using either solution like LVS (Linux Virtual Server) http://www.linuxvirtualserver.org or load-balancing dedicated equipment such as Radware Cache Server Director (or Web Server Director), F5 BIG IP, Nortel Alteon, Cisco (I wouldn't recommend since they are not good in this market, really). The thing is you have an equipment (either special switchs or linux servers) that dispatches the incoming requests to real servers. The load-balancer know the states of each real servers to prevent dispatching requests to an overloaded server. And to avoir single point of failure, the dispatcher is doubled in an active/passive way. Basically have to servers service the caching, but they must look like one server ... I am sure I sound stupid, but just to make sure. I hope it will help you. Sincerely yours, Mathieu.
RE: [squid-users] load balancing
ah ha!well do a search for Wccp, and do not just do it, implement it. You can also use LVS thats work goods as well. --- Chris Perreault [EMAIL PROTECTED] wrote: Do a search for round robin dns multiple a records clustering redundancy failover. That should give good coverage on the techniques Also...visiting www.squid-cache.org and searching for failover will help. Chris Perreault -Original Message- From: Gert Brits [mailto:[EMAIL PROTECTED] Sent: Thursday, July 22, 2004 7:40 AM To: [EMAIL PROTECTED] Org Subject: [squid-users] load balancing Hi All Please could I have either some ideas or solutions to the following: I work for a BIG company, providing internet services for many customers. I am planning to install a load balancing plan for our cache server, which will obviously load balance, but will also have a redundant feature. What can I do this with? Basically have to servers service the caching, but they must look like one server ... I am sure I sound stupid, but just to make sure. Regards Gert Brits = Regards, Mohsin Khan CCNA ( Cisco Certified Network Associate 2.0 ) http://portal.aaghaz.net (Your attention is needed) Happy is the one who can smile __ Do you Yahoo!? Yahoo! Mail is new and improved - Check it out! http://promotions.yahoo.com/new_mail
RE: [squid-users] load balancing
Check out http://naragw.sharp.co.jp/sps/. I think that this is the most simple and effective way as long as you don't mind using proxy.pac files and/or WPAD. Your other option is to use LVS w/ keepalived and ICP. Regards, David Chris Perreault [EMAIL PROTECTED] 7/22/2004 8:02:00 AM Do a search for round robin dns multiple a records clustering redundancy failover. That should give good coverage on the techniques Also...visiting www.squid-cache.org and searching for failover will help. Chris Perreault -Original Message- From: Gert Brits [mailto:[EMAIL PROTECTED] Sent: Thursday, July 22, 2004 7:40 AM To: [EMAIL PROTECTED] Org Subject: [squid-users] load balancing Hi All Please could I have either some ideas or solutions to the following: I work for a BIG company, providing internet services for many customers. I am planning to install a load balancing plan for our cache server, which will obviously load balance, but will also have a redundant feature. What can I do this with? Basically have to servers service the caching, but they must look like one server ... I am sure I sound stupid, but just to make sure. Regards Gert Brits -- SPAM filtering options: Spam is defined as unsolicited commercial email. -- ITS has provided the following optional links to train our Anti-Spam system. If the subject contains a ___ then this email was identified as possible Spam. If you wish to identify this email as SPAM then click here. Spam: http://ns1b.hillsboroughcounty.org/canit/b.php?c=si=378368m=29cf55a9fe1e If you wish to identify this email as not being Spam then click here. Not spam: http://ns1b.hillsboroughcounty.org/canit/b.php?c=ni=378368m=29cf55a9fe1e --
Re: [squid-users] Load Balancing with Cache_Peers
On Wed, 14 Jul 2004, Kent, Mr. John (Contractor) wrote: The part I don't understand is the redirection: the page says: To implement this solution your redirector script must output a URL where the hostname part of the URL is a keyword which describes a pool of backend servers, such as http://backendpool/VirtualHostBase/http/www.example.com:80/a/b/c Note that the hostname part of the URL is not a real host; it is a keyword that will be used in squid's configuration. I want to try to take advantage of that sophisticated logic. VirtualHostBase is a a Zope specific keyword. What should my redirectors return to call an apache backend specified by the backendpool or can it even be done? This is just Zope garbage and can be ignored for Apache. You do not need a redirector, just cache_peer. Regards Henrik
[squid-users] Load Balancing with Cache_Peers
Greeting Squid Gurus, I read an interesting article on Load Balancing in Zope with Squid as an accelerator. http://www.zope.org/Members/htrd/howto/squid I wanted to try it using Apache servers as a backend instead of Zope The problem is the article didn't quite have enough info for me to figure out how to do it (I did send the author an email) so was hoping someone in this list could fill me in. Squid can also make http requests to other caches, which Zope can understand. Squid contains some sophisticated logic for managing connections to a pool of other caches, and these features prove to be useful for managing a pool of backend Zope servers too According to the page I just need to add to my squid.conf (replaced their backendzope with backendApacheName) cache_peer backendApacheName1.dmz.example.com parent 8080 8080 no-digest no-netdb-exchange round-robin cache_peer backendApacheName2.dmz.example.com parent 8080 8080 no-digest no-netdb-exchange round-robin acl in_backendpool dstdomain backendpool cache_peer_access backendApacheName1.dmz.example.com allow in_backendpool cache_peer_access backendApacheName1.dmz.example.com deny all cache_peer_access backendApacheName2.dmz.example.com allow in_backendpool cache_peer_access backendApacheName2.dmz.example.com deny all never_direct allow all The never_direct line will ensure that Squid does not try to resolve the backendpool 'host' keyword as if it was a real host name, to connect to it if all the peers are down. You may need a more sophisticated never_direct acl if you have some backend servers which are not presented as peers. The configuration above assumes that the two backend zopes are providing http and ICP on port 8080. To use ICP you will need to enable it with the --icp command line switch, and you will need to some patches http://www.zope.org/Members/htrd/icp for Zope versions before 2.6. Alternatively include the no-query directive in the cache_peer lines. The part I don't understand is the redirection: the page says: To implement this solution your redirector script must output a URL where the hostname part of the URL is a keyword which describes a pool of backend servers, such as http://backendpool/VirtualHostBase/http/www.example.com:80/a/b/c Note that the hostname part of the URL is not a real host; it is a keyword that will be used in squid's configuration. I want to try to take advantage of that sophisticated logic. VirtualHostBase is a a Zope specific keyword. What should my redirectors return to call an apache backend specified by the backendpool or can it even be done? Thank you, John Kent
Re: [squid-users] Load balancing multiple Squid servers
On Wed, 17 Dec 2003, Cavanagh, Kevin B wrote: Hi there, Please forgive me if this question has been asked/answered before (I searched the FAQs but quickly became too confused by all the various postings regarding load balancing, etc). We currently have six RedHat Linux V8.0 servers running Squid V2.5 Stable in the following parent/child proxy chain: User U Iprism URL filters Squid caching (child) HTTP Anti-virus (parent) Internet We have six of each - with each currently having a one-to-one relationship (ie. IPRISM1P SQUID1 AV1 INET, etc).We would like to create a load-balanced situation here if at all possible. probably the only part we can help you with is the way that SQUID selects which AV parent to send a request to. One approach is to make a DNS name or /etc/hosts entry for the AV servers that has all their IP addresses. Then you put a single line in squid.conf: cache_peer av-servers.example.com parent 0 no-query Another approach is to list all servers separately and use the round-robin option cache_peer 172.16.0.1 parent 0 no-query round-robin cache_peer 172.16.0.2 parent 0 no-query round-robin cache_peer 172.16.0.3 parent 0 no-query round-robin cache_peer 172.16.0.4 parent 0 no-query round-robin cache_peer 172.16.0.5 parent 0 no-query round-robin cache_peer 172.16.0.6 parent 0 no-query round-robin Yet another approach is to use CARP: cache_peer 172.16.0.1 parent 0 no-query carp-load-factor=0.16 cache_peer 172.16.0.2 parent 0 no-query carp-load-factor=0.16 cache_peer 172.16.0.3 parent 0 no-query carp-load-factor=0.17 cache_peer 172.16.0.4 parent 0 no-query carp-load-factor=0.17 cache_peer 172.16.0.5 parent 0 no-query carp-load-factor=0.17 cache_peer 172.16.0.6 parent 0 no-query carp-load-factor=0.17 Duane W.
[squid-users] Load balancing multiple Squid servers
Hi there, Please forgive me if this question has been asked/answered before (I searched the FAQs but quickly became too confused by all the various postings regarding load balancing, etc). We currently have six RedHat Linux V8.0 servers running Squid V2.5 Stable in the following parent/child proxy chain: User U Iprism URL filters Squid caching (child) HTTP Anti-virus (parent) Internet We have six of each - with each currently having a one-to-one relationship (ie. IPRISM1P SQUID1 AV1 INET, etc).We would like to create a load-balanced situation here if at all possible. Could someone be kind enough to point us in the right direction??? TIA! Kevin Cavanagh
Re: [squid-users] Load Balancing Content Types
On Sun, 21 Sep 2003, Darryn Lowe wrote: Therefore, is it possible to have Squid change parent proxies based on the content type of the requested URL? How do I do this? This you can't, as the content type is not known until the reply is seen. What you can do is to select parent proxy based on an estimate of what the content type will be based on the extension of the URL (i.e. http://www.example.com/some/song.mp3 is quite likely a mp3 file). For this purpose see the urlpath_regex acl and the cache_peer_access directive. Regards Henrik
[squid-users] Load Balancing Content Types
Hi, I'm new to many of the options within Squid, so I would appreciate any help with the following scenario. I have network wherein the squid proxy connects to the Internet via two parent proxies. /- ISDN Parent Proxy #1 | Squid ---+ Proxy | \- ADSL Parent Proxy #2 Parent Proxy #1 (over the IDSN line) is monitored by an organisation-wide content filtering process. Although the simple solution is to simply use this link alone, the cost required to upgrade it to the required capacity is prohibitive. Parent Proxy #2 (over the ADSL line) is a high capacity multi-megabit connection that, while lacking any filtering capability, is cheap and fast. Hence, what I would like to do is have Parent Proxy #2 used for any non-textual content (e.g. image files, audio files, etc) while Parent Proxy #1 is used for generic HTML. Therefore, is it possible to have Squid change parent proxies based on the content type of the requested URL? How do I do this? Finally, are there any other options to balance the load over these two connections? The ideal solution would appear to have all requests go through to Parent Proxy #1 and then, when the squid proxy received a HTTP response that suggested the desired URL is accessible, the request would be dropped and then re-made via Parent Proxy #2. Any and all comments appreciated. Thanks, Darryn.
[squid-users] Load Balancing
Hi, I have two instances of squid-2.5-STABLE3 running on same dual-processor machine under RH AS 2.1 (2.4.9-e.25). Everything works great. I would like to optimize hit ratio and have a following questions: Is it possible to configure squid-es with a cross referencing access to cache directories, f.e.: Squid #1 cache_dir aufs /cache1 3 46 256 cache_dir aufs /cache2 3 46 256 read-only Squid #2 cache_dir aufs /cache1 3 46 256 read-only cache_dir aufs /cache2 3 46 256 I haven't tried to do it. Any suggestions? Regards, Yuri N. Fominov
Re: [squid-users] Load Balancing
On Sat, 2003-08-16 at 22:59, Yuri N. Fominov wrote: Hi, I have two instances of squid-2.5-STABLE3 running on same dual-processor machine under RH AS 2.1 (2.4.9-e.25). Everything works great. I would like to optimize hit ratio and have a following questions: Is it possible to configure squid-es with a cross referencing access to cache directories, f.e.: No. Use ICP, or cache digests between the instances. Rob -- GPG key available at: http://members.aardvark.net.au/lifeless/keys.txt. signature.asc Description: This is a digitally signed message part
[squid-users] load balancing rproxy?
I am a very happy user of the rproxy function of squid for about 2 years now. I was wondering: What if I place the rproxy somewhere at a central co-location, I have about 3 internet connections in the firm (with different ISP's) If I make my host available using all these lines, can I make squid to do the load balancing over these lines? Of make it somehowe now that if one line (ip address) fails it has to use the other?
[squid-users] load balancing
the main drawback of load balancing between peers is that squid never tries to determine how much of bandwitch left on other parents, just query first one. IMO besides connect-timeout there should be use-band option, allowing squid to not use certain parent if there are connections pending to it using x of bandwitch. this would introduce true load balancing, as number of connections isn't good measure. in addition parent cache can occassionally return it's bandwidth report , and databases could hold ping reply times and troughput to sites (servers) as 'cost' . -- --
Re: [squid-users] Load balancing on single machine
Dusan Djordjevic wrote: Hi all, I plan to install few instances of Squid on one multiprocessor box and balance load between them. I plan to use LinuxVirtualServer for it. Do someone have that kind of solution ? What load balancing you suggest ? Any other recommendation ? Do you have an idea of the sustained http reqs/sec , your squid has to deal with ? On design terms I would question whether this setup is meaningfull,since only one box is being used. Single point of failure issues e.d. would probably lead to think that at least 2 boxes, with one virtual address (server) would be better. M. Thanks in advance... - Eng. Dusan Djordjevic (RHCE) PlanetSky Ltd. Tel: +357 22454896*Fax: +357-22518022 http://www.planetsky.com [EMAIL PROTECTED] -- 'Love is truth without any future. (M.E. 1997)
Re: [squid-users] Load balancing on single machine
Dusan Djordjevic wrote: Monday 07 July 2003 10:56, Marc Elsen: I plan to install few instances of Squid on one multiprocessor box and balance load between them. I plan to use LinuxVirtualServer for it. Do someone have that kind of solution ? What load balancing you suggest ? Any other recommendation ? Do you have an idea of the sustained http reqs/sec , your squid has to deal with ? Yes. I need specific non-caching proxy. There are 3000-5000 requests per second. Currently I have one 4CPU box i would like to deploy. Hm, I think I read that squid can give you a max. of about 300 reqs/sec. As stated before on the list squid on itself can not make use of more then one cpu. So I think in a virtual server setup, you may be better off with 4 separate boxes (probably), I think it would give you more flexibility and squid service uptime, if one box is down, for instance. M.
Re: [squid-users] Load balancing on single machine
Masood Ahmad Shah [EMAIL PROTECTED] writes: hmmm but what do u think if the proxy is transparent ..:) [...] The trick is the same. Just do two rules, one with source -s 0.0.0.0/0.0.0.1 redirecting to the first instance of the transparent proxy, the other with \! -s 0.0.0.0/0.0.0.1 redirecting to the other instance. -- kinkie (kinkie-squid [at] kinkie [dot] it) Random fortune, unrelated to the message: Yow! Am I in Milwaukee?
Re: [squid-users] Load balancing on single machine
In my sense if one proxy will down another will not take it's place because iptable rules are redirecting packets to both port and port instance is not running.. so what type of load balancing .:) Better to run Linux clustering that is much better then things like that... -- Best Regs, Masood Ahmad Shah System Administrator ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ | * * * * * * * * * * * * * * * * * * * * * * * * | Fibre Net (Pvt) Ltd. Lahore, Pakistan | Tel: +92-42-6677024 | Mobile: +92-300-4277367 | http://www.fibre.net.pk | * * * * * * * * * * * * * * * * * * * * * * * * ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ Unix is very simple, but it takes a genius to understand the simplicity. (Dennis Ritchie) All I want is a few minutes alone with the source code for the universe and a quick recompile. - Original Message - From: Kinkie [EMAIL PROTECTED] To: Masood Ahmad Shah [EMAIL PROTECTED] Cc: Marc Elsen [EMAIL PROTECTED]; Dusan Djordjevic [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Monday, July 07, 2003 2:34 PM Subject: Re: [squid-users] Load balancing on single machine | Masood Ahmad Shah [EMAIL PROTECTED] writes: | | hmmm but what do u think if the proxy is transparent ..:) | | [...] | | The trick is the same. Just do two rules, one with source | | -s 0.0.0.0/0.0.0.1 | | redirecting to the first instance of the transparent proxy, the other with | | \! -s 0.0.0.0/0.0.0.1 | | redirecting to the other instance. | | -- | kinkie (kinkie-squid [at] kinkie [dot] it) | Random fortune, unrelated to the message: | Yow! Am I in Milwaukee? |
Re: [squid-users] Load balancing on single machine
Masood Ahmad Shah [EMAIL PROTECTED] writes: In my sense if one proxy will down another will not take it's place because iptable rules are redirecting packets to both port and port instance is not running.. so what type of load balancing .:) Load balancing _is_ in effect. It's the health-checking that's not. For that you'd have to concot some script modifying the iptables rules depending on service availability. I'll leave that as an exercise for the readers. Better to run Linux clustering that is much better then things like that... It's just done in a different way. BTW, I tried to use linuxvirtualserver on the same host, but for some reason it didn't really work. -- kinkie (kinkie-squid [at] kinkie [dot] it) Random fortune, unrelated to the message: Successful and fortunate crime is called virtue. - Seneca
[squid-users] load balancing HTTP servers using Squid
Hi, I'm searching for a software load balancer for HTTP servers. Scenario: We have two web servers in Germany and two in the USA for german content. For US/english content there are two web servers in the USA and two in Germany. The setup is similar, so I'll concentrate to a single case. Dream: Requests should get dynamically balanced to the two german web servers. If one fails, the second gets each and every request. If both fail, users will be served by the US servers using HTTP redirects or reverse proxying. Internet | | german load balancer --- US load balancer | | | | firewall firewall /\ /\ / \ / \ www1 www2 www3 www4 (The load balancer never fail ;-)) Steps to a solution: http://devel.squid-cache.org/rproxy/ seems to be a good starting point. What it's state? Can Squid-2.5-STABLE2 be used for this setup? Is Squid-3.0-DEV geared towards these requirements? http://www.squid-cache.org/mail-archive/squid-dev/200010/0321.html has a nice patch for 2.3-STABLE4, but how far has development gone? Regards Bernie
