date:20200113

[squid-users] TCP incoming requests Traffic Normalization

2020-01-13 Thread Ahmad Alzaeem


Hello Folks .

I have about 10x sources or different ip addresses  and sending requests to 
squid  .

imagine we have 10 servers and sending burst in sometimes due to nature of 
Traffic  ….i have a sensitive APP on squid that must be equalized to handle 
only 50 req/sec . “ No more “

i just want to equalize all incoming requests which can be in some seconds 60  
, 40 , 90 , 100 , 50 to have steady 50 req/sec on squid equally and even if we 
need to delay some packs  its ok , just keep squid handle 50 req/sec  of those 
incoming requests no  more .

i know squid can limit connections and Drop connections above threshold , but i 
need only to discipline and Buffer and try to decrease dropped requests as 
possible and normalize all incoming requests to be steady 50 req/sec inside 
squid whatever there is burst outside or higher than 50 .

So again just need to apply that on “ new requests “ not on already “ 
established “ connections .

Let me know Guys if squid can do something like that or we need 3rd party 
outside squid .


Kind regards 

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] [TECHNICAL QUESTION] try to resolve 403 error for specific website

2020-01-13 Thread killpilot

hi squid community,

sorry for my bad english, i french, i try do my best for explain
cleary my issue.

i have a pfsense with squid plugin. the plugin contain :
squidclamav-6.16
squid_radius_auth-1.10
squid-3.5.27_3
c-icap-modules-0.5.3_1

my squid is config for transparent proxy for http only.

for on game Star Citizen i have a issue with voip feature, when i try
launch voip connection, she failed.

in my squid log i see this entry
1578684384.329 237 192.168.2.2 TCP_MISS/403 270 GET
http://foip-v02.robertsspaceindustries.com/ -
ORIGINAL_DST/35.153.171.151 text/html
1578684385.507 165 192.168.2.2 TCP_MISS/403 270 GET
http://foip-v02.robertsspaceindustries.com/ -
ORIGINAL_DST/35.153.171.151 text/html

when i disable squid, all working fine.
my squid conf file is :

-- My conf file ---
# This file is automatically generated by pfSense
# Do not edit manually !

http_port 192.168.2.1:3128
http_port 192.168.4.1:3128
http_port 192.168.8.1:3128
http_port 127.0.0.1:3128 intercept
icp_port 0
digest_generation off
dns_v4_first off
pid_filename /var/run/squid/squid.pid
cache_effective_user squid
cache_effective_group proxy
error_default_language fr
icon_directory /usr/local/etc/squid/icons
visible_hostname localhost
cache_mgr 
access_log /var/squid/logs/access.log
cache_log /var/squid/logs/cache.log
cache_store_log none
netdb_filename /var/squid/logs/netdb.state
pinger_enable on
pinger_program /usr/local/libexec/squid/pinger

logfile_rotate 7
debug_options rotate=7
shutdown_lifetime 3 seconds
# Allow local network(s) on interface(s)
acl localnet src  192.168.2.0/24 192.168.4.0/24 192.168.8.0/24
forwarded_for on
httpd_suppress_version_string on
uri_whitespace strip


cache_mem 64 MB
maximum_object_size_in_memory 256 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
minimum_object_size 0 KB
maximum_object_size 10 MB
cache_dir ufs /var/squid/cache 1024 16 256
offline_mode off
cache_swap_low 90
cache_swap_high 95
acl donotcache dstdomain "/var/squid/acl/donotcache.acl"
cache deny donotcache
cache allow all
# Add any of your own refresh_pattern entries above these.
refresh_pattern ^ftp:1440  20%  10080
refresh_pattern ^gopher:  1440  0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0  0%  0
refresh_pattern .0  20%  4320

#Remote proxies

# Setup some default acls
# ACLs all, manager, localhost, and to_localhost are predefined.
acl allsrc src all
acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 9443
3128 3129 1025-65535
acl sslports port 443 563 9443

acl purge method PURGE
acl connect method CONNECT

# Define protocols used for redirects
acl HTTP proto HTTP
acl HTTPS proto HTTPS
acl unrestricted_hosts src "/var/squid/acl/unrestricted_hosts.acl"
acl whitelist dstdom_regex -i "/var/squid/acl/whitelist.acl"
http_access allow manager localhost

http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !safeports
http_access deny CONNECT !sslports

# Always allow localhost connections
http_access allow localhost

request_body_max_size 0 KB
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_initial_bucket_level 100
# Do not throttle unrestricted hosts
delay_access 1 deny unrestricted_hosts
delay_access 1 allow allsrc

# Reverse Proxy settings
acl rvm_uri_proxmox url_regex -i proxmox.killpilot.fr
never_direct allow rvm_uri_proxmox
http_access allow rvm_uri_proxmox


# Custom options before auth
acl voip_rsi dstdomain .robertsspaceindustries.com
always_direct allow voip_rsi
cache deny voip_rsi
http_access allow voip_rsi

# These hosts do not have any restrictions
http_access allow unrestricted_hosts
# Always allow access to whitelist domains
http_access allow whitelist
# Setup allowed ACLs
# Allow local network(s) on interface(s)
http_access allow localnet
# Default block all to be sure
http_access deny allsrc

icap_enable on
icap_send_client_ip off
icap_send_client_username off
icap_client_username_encode off
icap_client_username_header X-Authenticated-User
icap_preview_enable on
icap_preview_size 1024

icap_service service_avi_req reqmod_precache
icap://127.0.0.1:1344/squid_clamav bypass=off
adaptation_access service_avi_req allow all
icap_service service_avi_resp respmod_precache
icap://127.0.0.1:1344/squid_clamav bypass=on
adaptation_access service_avi_resp allow all
-- END My conf file ---


i try to add this block
acl voip_rsi dstdomain .robertsspaceindustries.com
always_direct allow voip_rsi
cache deny voip_rsi
http_access allow voip_rsi

but, not resolved my issue,

i also try add this conf into this file :
my file /var/squid/acl/donotcache.acl  contain :
robertsspaceindustries.com

my file /var/squid/acl/unrestricted_hosts.acl contain my pc IP
192.168.2.2/32

my file /var/squid/acl/whitelist.acl contain
^.*\.robertsspaceindustries.com

same result, failed i don't understand why the request are denied .
from my pc i try with curl command the result is :

curl -vvv -x http://192.168.2.1

[squid-users] Squid configuration cache_peer does not take effect?

2020-01-13 Thread yohan83942

The configuration is as follows:

```shell
# Squid normally listens to port 3128
always_direct allow all
ssl_bump bump all
sslproxy_cert_error allow all
http_port 3128 ssl-bump cert=/etc/squid/squid.pem key=/etc/squid/squid.pem
generate-host-certificates=on options=NO_SSLv2
#http_port 3128

cache_peer 127.0.0.1 parent 10809 0 no-query
never_direct allow all
```

I tested it and found out that the `127.0.0.1:10809` proxy was not taken?

How to solve? 

However, when I changed it to the following, it worked. But then Squid
cannot cache https.

```shell
http_port 3128

cache_peer 127.0.0.1 parent 10809 0 no-query
never_direct allow all
```

Why is that?

Full configuration

```shell
#
# Recommended minimum configuration:
#

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7   # RFC 4193 local private network range
acl localnet src fe80::/10  # RFC 4291 link-local (directly plugged)
machines

acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# And finally deny all other access to this proxy
http_access deny all

# Squid normally listens to port 3128
always_direct allow all
ssl_bump bump all
sslproxy_cert_error allow all
http_port 3128 ssl-bump cert=/etc/squid/squid.pem key=/etc/squid/squid.pem
generate-host-certificates=on options=NO_SSLv2
#http_port 3128

cache_peer 127.0.0.1 parent 10809 0 no-query
never_direct allow all

# Uncomment and adjust the following to add a disk cache directory.
cache_dir ufs /var/cache/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /var/cache/squid

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp:   144020% 10080
refresh_pattern ^gopher:14400%  1440
refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
refresh_pattern .   0   20% 4320

dns_nameservers 8.8.8.8
```



--
Sent from: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Squid configuration cache_peer does not take effect?

2020-01-13 Thread yohan83942

The configuration is as follows:

```shell
# Squid normally listens to port 3128
always_direct allow all
ssl_bump bump all
sslproxy_cert_error allow all
http_port 3128 ssl-bump cert=/etc/squid/squid.pem key=/etc/squid/squid.pem
generate-host-certificates=on options=NO_SSLv2
#http_port 3128

cache_peer 127.0.0.1 parent 10809 0 no-query
never_direct allow all
```

I tested it and found out that the `127.0.0.1:10809` proxy was not taken?

How to solve? 

However, when I changed it to the following, it worked. But then Squid
cannot cache https.

```shell
http_port 3128

cache_peer 127.0.0.1 parent 10809 0 no-query
never_direct allow all
```

Why is that?

Full configuration

```shell
#
# Recommended minimum configuration:
#

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7   # RFC 4193 local private network range
acl localnet src fe80::/10  # RFC 4291 link-local (directly plugged)
machines

acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# And finally deny all other access to this proxy
http_access deny all

# Squid normally listens to port 3128
always_direct allow all
ssl_bump bump all
sslproxy_cert_error allow all
http_port 3128 ssl-bump cert=/etc/squid/squid.pem key=/etc/squid/squid.pem
generate-host-certificates=on options=NO_SSLv2
#http_port 3128

cache_peer 127.0.0.1 parent 10809 0 no-query
never_direct allow all

# Uncomment and adjust the following to add a disk cache directory.
cache_dir ufs /var/cache/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /var/cache/squid

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp:   144020% 10080
refresh_pattern ^gopher:14400%  1440
refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
refresh_pattern .   0   20% 4320

dns_nameservers 8.8.8.8
```



--
Sent from: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid configuration cache_peer does not take effect?

2020-01-13 Thread info

> The configuration is as follows:
> 
> ```shell
> # Squid normally listens to port 3128
> always_direct allow all
> ssl_bump bump all
> sslproxy_cert_error allow all
> http_port 3128 ssl-bump cert=/etc/squid/squid.pem key=/etc/squid/squid.pem
> generate-host-certificates=on options=NO_SSLv2
> #http_port 3128
> 
> cache_peer 127.0.0.1 parent 10809 0 no-query
> never_direct allow all
> ```
> 

Hi,

as far as i know, cache_peer seems to be mostly not supported with ssl_bump 
before Squid v5 (not stable yet), see here: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/Non-transparent-proxy-with-cache-peer-and-ssl-bump-tp4687620p4687622.html
 

Had the same issue and gave up, will wait for Squid v5 becomes productive 
usable.

All the best
Schroeffu
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid configuration cache_peer does not take effect?

2020-01-13 Thread yohan83942

I have read this article before and it has been a long time. The problem
persists and I'm surprised.
Do you know when the v5 version will be released? Already 2020.

Is there a solution?
I just want a server that can cache https and access the Internet through a
proxy like `cache_peer`?



--
Sent from: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] TCP incoming requests Traffic Normalization

2020-01-13 Thread Alex Rousskov

On 1/13/20 5:36 AM, Ahmad Alzaeem wrote:

> just need to apply that on “ new requests “ not on already “ established “ 
> connections

You can probably do this using an external ACL. The program implementing
your external ACL can delay incoming requests to make sure that the
aggregate served request rate is at most 50/s. You will need to write
that program or find something suitable on the web.

For more details, look for "acl aclname external" and perhaps "external
ACL" in squid.conf.documented. The API for the external ACL program
(called "helper") is described at
https://wiki.squid-cache.org/Features/AddonHelpers#Access_Control_.28ACL.29

N.B. One connection may carry many HTTP requests. To implement the limit
correctly, you will need to clarify whether the limit should apply to
new requests on established connections. The external ACL helper can
support any answer to that question, but its code will be different
depending on your answer.

Alex.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Squid memory usage increase overtime, proxies slows down

2020-01-13 Thread squdbuff

Hello guys

Running squid on ubuntu 16

When I first start the program, the proxies are at a good speed (below
100ms)

after about 5 mins then memory usage of squid increases greatly, then the
proxies slow down to 1000ms+

my access log is getting spammed by the follow message ( I have no idea what
this means) I dont have a user named kaiz9n9d9

1578959115.266  0 216.115.184.251 TCP_DENIED/407 4130 CONNECT
www.supremenewyork.com:443 kaiz9n9d9 HIER_NONE/- text/html
1578959115.270  0 216.115.184.251 TCP_DENIED/407 4020 CONNECT
www.supremenewyork.com:443 - HIER_NONE/- text/html
1578959115.272  0 216.115.184.251 TCP_DENIED/407 4130 CONNECT
www.supremenewyork.com:443 kaiz9n9d9 HIER_NONE/- text/html
1578959115.276  0 216.115.184.251 TCP_DENIED/407 4020 CONNECT
www.supremenewyork.com:443 - HIER_NONE/- text/html
1578959115.277  0 216.115.184.251 TCP_DENIED/407 4130 CONNECT
www.supremenewyork.com:443 kaiz9n9d9 HIER_NONE/- text/html
1578959115.281  0 216.115.184.251 TCP_DENIED/407 4020 CONNECT
www.supremenewyork.com:443 - HIER_NONE/- text/html


Currently I am running "service squid reload", every 300 seconds which keeps
the proxies at a good speed

But this is not a permanent fix

Can someone please offer your advise for this issue?

Thanks!



--
Sent from: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid memory usage increase overtime, proxies slows down

2020-01-13 Thread squdbuff

When I first start squid memory usage is around 24mb, then after 5 mins
increased to 1GB!

Also, the access log size increases by 500 kb every second, so I have
disabled access log for now



--
Sent from: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid configuration cache_peer does not take effect?

2020-01-13 Thread Amos Jeffries

On 14/01/20 4:45 am, yohan83942 wrote:
> I have read this article before and it has been a long time. The problem
> persists and I'm surprised.
> Do you know when the v5 version will be released? Already 2020.

I am bundling 5.0.1 (beta) later today. As for when it's stable that
will depend on what bugs get found and how hard they are to fix.
Hopefully later this year or early next.

You do not need a Squid version to be stable to use it. Just expect some
bugs.

Code for 5.0.0 (alpha) is available right now at
 if you are able to self-build.

> 
> Is there a solution?

Only the feature in v5.

There is a hacks, but that requires the peer to be a Squid with SSL-Bump
and requires some tricky config at the peer. It is unlikely to be useful
unless you are admin of both proxies.

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] TCP incoming requests Traffic Normalization

[squid-users] [TECHNICAL QUESTION] try to resolve 403 error for specific website

[squid-users] Squid configuration cache_peer does not take effect?

[squid-users] Squid configuration cache_peer does not take effect?

Re: [squid-users] Squid configuration cache_peer does not take effect?

Re: [squid-users] Squid configuration cache_peer does not take effect?

Re: [squid-users] TCP incoming requests Traffic Normalization

[squid-users] Squid memory usage increase overtime, proxies slows down

Re: [squid-users] Squid memory usage increase overtime, proxies slows down

Re: [squid-users] Squid configuration cache_peer does not take effect?

10 matches

Site Navigation

Mail list logo

Footer information