[squid-users] caching apt package lists/Raspbian
Recently upgraded to Raspbian Buster and squid 4.6. Since then I am unable to cache the Packages.xz that apt uses. The various other Pis using this proxy all end up downloading the 30MB Packages.xz every time. Does anyone have any suggestions on how to get it to cache? Cheers MarkJ squid -v Squid Cache: Version 4.6 Service Name: squid Raspbian linux access.log 1563597855.786 605 192.168.1.73 TCP_REFRESH_UNMODIFIED/200 15306 GET http://raspbian.raspberrypi.org/raspbian/dists/buster/InRelease - HIER_DIRECT/93.93.128.193 - 1563597855.811 620 192.168.1.73 TCP_REFRESH_UNMODIFIED/200 25429 GET http://archive.raspberrypi.org/debian/dists/buster/InRelease - HIER_DIRECT/93.93.128.133 - 1563597857.486 620 192.168.1.73 TCP_REFRESH_UNMODIFIED/200 205801 GET http://archive.raspberrypi.org/debian/dists/buster/main/binary-armhf/Packages.gz - HIER_DIRECT/93.93.128.133 application/x-gzip 1563597936.436 80026 192.168.1.73 TCP_MISS_ABORTED/200 2641974 GET http://raspbian.raspberrypi.org/raspbian/dists/buster/main/binary-armhf/Packages.xz - HIER_DIRECT/93.93.128.193 application/x-xz config file acl localnet src 192.168.1.0/24 # internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines acl l500-020b src 192.168.1.20 acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT acl ads dstdomain .ad1.pamedia.com.au acl ads dstdomain .ad3.pamedia.com.au acl ads dstdomain .adevents.com.au acl ads dstdomain .adinfinity.com.au acl ads dstdomain .ads.excitehome.net.au acl ads dstdomain .ads.fairfax.com.au acl ads dstdomain .ads.godaddy.com acl ads dstdomain .ads.google.com acl ads dstdomain .ads.ninemsn.com.au acl ads dstdomain .ads.optusnet.com.au acl ads dstdomain .ads.property.com.au acl ads dstdomain .ads.youtube.com acl ads dstdomain .adserver.news.com.au acl ads dstdomain .au.adserver.yahoo.com acl ads dstdomain .doubleclick.net acl ads dstdomain .googleadservices.com acl ads dstdomain .zoomdirect.com.au acl malware dstdomain am10.ru acl malware dstdomain deepspacer.com acl malware dstdomain trafficconverter.biz acl malware dstdomain .eu.interia.pl acl malware dstdomain .expo9.exponential.com acl malware dstdomain .flashtalking.com acl malware dstdomain .funad.co.kr acl malware dstdomain .luckytime.co.kr acl malware dstdomain .trafficholder.com acl malware2 dst 96.43.128.194 acl hiddenwasp dst 103.206.122.245 acl hiddenwasp dst 103.206.123.13 acl hiddenwasp2 dstdomain http://103.206.123.13 http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access deny ads http_access deny malware http_access deny malware2 http_access deny hiddenwasp http_access deny hiddenwasp2 http_access allow l500-020b manager http_access deny manager http_access allow localnet http_access allow localhost http_access deny all http_port 3128 cache_mem 448 MB maximum_object_size 320 MB memory_replacement_policy lru cache_replacement_policy heap LFUDA cache_dir aufs /var/spool/squid 18432 32 256 quick_abort_min -1 KB client_request_buffer_max_size 128 KB coredump_dir /var/spool/squid access_log /var/log/squid/access.log squid cache_log /var/log/squid/cache.log logfile_rotate 7 netdb_filename none refresh_pattern (\.deb|\.udeb)$ 1440 80% 10080 refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 host_verify_strict on max_filedescriptors 1200 dns_v4_first on pinger_enable off shutdown_lifetime 5 seconds ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Possible to user reply_header_add directive with acl random access list ?
Alex you have been helpful a-lot . i would appreciate your help & Amos for what you provided . Thanks for your kind support . you have simplified all what i need . Kind regards > On 19 Jul 2019, at 23:03, Alex Rousskov > wrote: > > reply_header_add Start "%___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Possible to user reply_header_add directive with acl random access list ?
On 7/19/19 2:54 PM, --Ahmad-- wrote: > say i have 10 ips > i want random external over them . > if external was ip1 , then “start header” should be A > if external was ip2 , then “start header” should be b > if external was ip3 , then “start header” should be c > if external was ip4 , then “start header” should be d I can suggest two options. The first one is a little simpler, but it uses actual IP addresses (e.g., "1.1.1.1") instead of IP address labels/pseudonyms (e.g. "A") for Start header values: # select one of ten IPs using a uniform random distribution tcp_outgoing_address 12.13.200.10 p1in10 tcp_outgoing_address 12.13.200.11 p1in9 ... tcp_outgoing_address 12.13.200.19 all # tell the client what IP our to-server connection originated from reply_header_add Start "%http://lists.squid-cache.org/listinfo/squid-users
[squid-users] Ubuntu 18 LTS repository for Squid 4.8 (rebuilt with sslbump support from sources in Debian unstable)
Greeting all, The online repository with latest Squid 4.8 (rebuilt from Debian unstable with sslbump support) for Ubuntu 18 LTS 64-bit is available at squid48.diladele.com. Github repo at https://github.com/diladele/squid-ubuntu contains the scripts we used to make this compilation (look for feature-squid-4.8-1 branch). Scripts for Ubuntu 16 will be updated in the near future. Hope you will find this helpful. Note that older repo of squid46.diladele.com will be taken down in two years. Best regards, Rafael Akchurin Diladele B.V. P.S. Here are simple instructions how to use the repo. For more information see readme at https://github.com/diladele/squid-ubuntu . # add diladele apt key wget -qO - http://packages.diladele.com/diladele_pub.asc | sudo apt-key add - # add repo echo "deb http://squid48.diladele.com/ubuntu/ bionic main" > /etc/apt/sources.list.d/squid48.diladele.com.list # update the apt cache apt-get update # install apt-get install squid-common apt-get install squid apt-get install squidclient -- Please take a look at another our project - DNS Safety filtering server. Sort of Web Safety implemented as DNS Server. Might be interesting in deployments where HTTPS decryption is not possible. https://dnssafety.io/ ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Possible to user reply_header_add directive with acl random access list ?
Alex .. indeed i asked many questions and you already solved me old issues . i do apologise for that Drop . here is what we are going to achieve . in simple : i want to have external random addressees from list of addresses . and in the same time i want a header like “start” header who can be sent from squid to Host with tag. say i have 10 ips i want random external over them . and i want single on each those 10 ips be sent back to Host. if external was ip1 , then “start header” should be A if external was ip2 , then “start header” should be b if external was ip3 , then “start header” should be c if external was ip4 , then “start header” should be d and so on . Thanks and again Guys you have been much helpful . Thanks > On 19 Jul 2019, at 16:08, Alex Rousskov > wrote: > > On 7/19/19 8:53 AM, --Ahmad-- wrote: > >> is there any way can i let header acl stop on the 1st MATCH ? > > Yes, your reply_header_add ACLs effectively stop on the first match, > using the annotation trick. That part of your configuration is probably > working. The primary problem is elsewhere. > > >> do you have any other thing can we do to achieve what im looking for >> based on my config below ? > > FWIW, I do not know what you are looking for. I even checked earlier > emails on this thread and could not find that information. Can you > (re)state your goals using the following template? > > "When Squid receives a client request with HTTP header X, I want Squid > to forward that request using outgoing TCP address Y, and then add HTTP > header Z to the response that Squid sends to the client." > > Replace X, Y, and Z with your actual requirements. Adjust as needed, > including removing any unnecessary parts. > > Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Possible to user reply_header_add directive with acl random access list ?
On 7/19/19 8:53 AM, --Ahmad-- wrote: > is there any way can i let header acl stop on the 1st MATCH ? Yes, your reply_header_add ACLs effectively stop on the first match, using the annotation trick. That part of your configuration is probably working. The primary problem is elsewhere. > do you have any other thing can we do to achieve what im looking for > based on my config below ? FWIW, I do not know what you are looking for. I even checked earlier emails on this thread and could not find that information. Can you (re)state your goals using the following template? "When Squid receives a client request with HTTP header X, I want Squid to forward that request using outgoing TCP address Y, and then add HTTP header Z to the response that Squid sends to the client." Replace X, Y, and Z with your actual requirements. Adjust as needed, including removing any unnecessary parts. Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Possible to user reply_header_add directive with acl random access list ?
Hi Guys , Thank you a lot for your cooperation . is there any way can i let header acl stop on the 1st MATCH ? do you have any other thing can we do to achieve what im looking for based on my config below ? Thanks > On 19 Jul 2019, at 13:04, Amos Jeffries wrote: > > To make the IP based on the "a" existence you have to ... base it on the > "a" - not on some random number. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Possible to user reply_header_add directive with acl random access list ?
On 19/07/19 6:49 pm, --Ahmad-- wrote: > > But may be im wrong with config and im open now to any suggestions to > change the config to get it working as i mentioned above with headers . > As I said at the end of my earlier mail: " You appear to have missed the fact that each check/test of the ACL uses a different randomly selected number. " These: > > reply_header_add start "a" !markedProcessed half1 markProcessed > > tcp_outgoing_address 12.13.200.10 half1 > ... contain two different check/test of the ACL called half1. One for reply_header_add, another one for tcp_outgoing_address. -> a random 1/5 of requests will have "Start: a" header added. -> a random 1/5 of requests will try to send from 12.13.200.10 IP address. The two sets likely do not overlap. Though since this is truly random - there is a 2.5% chance that any request might *look* like what you are seeking. To make the IP based on the "a" existence you have to ... base it on the "a" - not on some random number. Amos ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Possible to user reply_header_add directive with acl random access list ?
Hi Alex . Strange: Your outgoing address decisions appear to be random, completely independent from your Start values. Is that what you want? yes , it suppose to have header as i configured the acls . > 12.13.200.13 --->D > 12.13.200.12 >C > 12.13.200.14 ——>E Not > 12.13.200.13 --> B > 12.13.200.14 --> a > 12.13.200.12 ---> E I see nothing in your configuration that would tie outgoing address to Start values. Where did you configure Squid to use "D" for .13 or vice versa? May im wrong in config , i thought that my config above like : ### dns_nameservers 1.0.0.1 acl markProcessed annotate_client processed=yes acl markedProcessed note processed yes # acl half1 random 1/5 reply_header_add start "a" !markedProcessed half1 markProcessed tcp_outgoing_address 12.13.200.10 half1 But may be im wrong with config and im open now to any suggestions to change the config to get it working as i mentioned above with headers . Thanks > On 19 Jul 2019, at 5:44, Alex Rousskov > wrote: > > Strange: Your outgoing address decisions appear to be random, completely > independent from your Start values. Is that what you want? ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
