[squid-users] squid and c-icap configuration

[Amiq Nahas]

Hi Guys,

This is what I want:
Browse the internet through a browser such that every url request goes
to squid proxy first and then the squid proxy sends it to c-icap
server. Finally the url should be logged into `/var/log/access.log`


This is what I have tried:

c-icap.conf:

PidFile /var/run/c-icap/c-icap.pid
CommandsSocket /var/run/c-icap/c-icap.ctl
Timeout 300
MaxKeepAliveRequests 100
KeepAliveTimeout 600
StartServers 3
MaxServers 10
MinSpareThreads 10
MaxSpareThreads 20
ThreadsPerChild 10
MaxRequestsPerChild  0
Port 1344
ServerAdmin you@your.address
ServerName YourServerName
TmpDir /var/tmp
MaxMemObject 131072
DebugLevel 1
Pipelining on
SupportBuggyClients off
ModulesDir /usr/local/lib/c_icap
ServicesDir /usr/local/lib/c_icap
TemplateDir /usr/local/share/c_icap/templates/
TemplateDefaultLanguage en
LoadMagicFile /usr/local/etc/c-icap.magic
RemoteProxyUsers off
RemoteProxyUserHeader X-Authenticated-User
RemoteProxyUserHeaderEncoded on
acl all src 0.0.0.0/0.0.0.0
LogFormat myFormat "%tl, %la %a %im %iu %is %huo"
ServerLog /usr/local/var/log/server.log
AccessLog /usr/local/var/log/access.log myFormat all
Service echo srv_echo.so


squid.conf

icap_enable on
icap_send_client_ip on
icap_send_client_username on
icap_client_username_header X-Authenticated-User
icap_preview_enable on
icap_preview_size 1024
icap_service service_req reqmod_precache bypass=1 icap://127.0.0.1:1344/request
adaptation_access service_req allow all
icap_service service_resp respmod_precache bypass=0
icap://127.0.0.1:1344/response
adaptation_access service_resp allow all


This is the result I have got by doing the above:

/var/log/access.log

19/May/2020:11:43:54 +0530, 127.0.0.1 127.0.0.1 UNKNOWN - 400 -
19/May/2020:11:45:15 +0530, 127.0.0.1 127.0.0.1 OPTIONS request 404 -
20/May/2020:00:44:56 +0530, 127.0.0.1 127.0.0.1 UNKNOWN - 400 -
20/May/2020:00:44:57 +0530, 127.0.0.1 127.0.0.1 UNKNOWN - 400 -
20/May/2020:00:45:00 +0530, 127.0.0.1 127.0.0.1 UNKNOWN - 400 -
20/May/2020:00:45:01 +0530, 127.0.0.1 127.0.0.1 UNKNOWN - 400 -
20/May/2020:00:45:15 +0530, 127.0.0.1 127.0.0.1 OPTIONS request 404 -
20/May/2020:00:48:15 +0530, 127.0.0.1 127.0.0.1 OPTIONS request 404 -
20/May/2020:00:51:15 +0530, 127.0.0.1 127.0.0.1 OPTIONS request 404 -
20/May/2020:00:54:15 +0530, 127.0.0.1 127.0.0.1 OPTIONS request 404 -
20/May/2020:00:57:15 +0530, 127.0.0.1 127.0.0.1 OPTIONS request 404 -


What am I doing wrong ? I should mention that

Thanks
Amiq
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Dumping sslbump'd decrytped http using icap protocol

[Eliezer Croitoru]

Hey, I think you might need fiddler not Squid. Eliezer Eliezer CroitoruTech SupportMobile: +972-5-28704261Email: ngtech1...@gmail.com From: ScottSent: Sunday, May 24, 2020 3:57 PMTo: squid-users@lists.squid-cache.orgSubject: [squid-users] Dumping sslbump'd decrytped http using icap protocol Hi, Can someone recommend an ICAP application that will allow me to dump the HTTP of a client-server conversation? I am doing some forensics on an app - I have sslbump configured correctly and I can get the traffic to c-icap (for example). I'd like to dump this to a text file. Is there a dump option for c-icap?  I couldn't find one. Thanks,Scott___squid-users mailing listsquid-users@lists.squid-cache.orghttp://lists.squid-cache.org/listinfo/squid-users 
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 4.4 https_port and ssl-bump : Fatal bungled line

[ben benml]

Hello,

Thank you for your prompt and precise answer.

Well I'm permit myself another question, sorry. If you have an opinion
about securing the authentification without https_port :
With a FreeIPA central users directory, what could be the best way to
secure/protect the  authentication process, the login/password.
Or more generally what could be the best options to secure the
login/password with only the http_port. So no directly encrypted traffic.

I was assuming https connection could secure the authentication process ..
but if ssl-dump  is really wanted, so I need another options to secure the
login/password.

Did you see my point / what I'm trying to talk about ?

Thank you in advance.

Regards,


Le lun. 25 mai 2020 à 12:26, Amos Jeffries  a écrit :

> On 25/05/20 9:59 pm, ben benml wrote:
> > Hello,
> >
> > I'm contacting you for some help.
> > I need to deploy a secure proxy based on Squid.
> >
> > I try to use https_port combined with sslbump. I get an error message
> > about a bungled line.
> >
> > The reasons I want to do this :
> > - secure connection between the client browser and the proxy server, so
> > using https_port to do it. encrypted  traffic in TLS between the client
> > and the server.
>
> Fine. Simply using https_port does that.
>
> > - secure login connection. So I need to use https_port to do this.
>
> Fine. Simply using https_port does that.
>
> > - Do ssl inspection of the traffic goeing through the proxy
>
> Squid does not yet support SSL-Bump decrypt of traffic already being
> decrypted for the secure proxy.
>
>
> Please see
> 
> if
> you want details.
>
> Amos
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Dumping sslbump'd decrytped http using icap protocol

[Scott]

On Mon, May 25, 2020 at 06:34:19PM +1200, Amos Jeffries wrote:
> On 25/05/20 12:56 am, Scott wrote:
> > Hi,
> > 
> > Can someone recommend an ICAP application that will allow me to dump the 
> > HTTP 
> > of a client-server conversation?
> > 
> > I am doing some forensics on an app - I have sslbump configured correctly 
> > and 
> > I can get the traffic to c-icap (for example).
> > 
> > I'd like to dump this to a text file.
> > 
> > Is there a dump option for c-icap?  I couldn't find one.
> > 
> 
> FYI; this action is illegal in a lot of places. Even answering your
> question can be quite risky.
> 
> 
> To perform traffic forensics you can use the Squid cache.log directly
> and not involve any insecure third-party software or communication
> dumps. See 
> for more details.
> 
> "debug_Options 11,2" is probably all you need.
> 
> 
> Amos
> 
Thanks,

I'm inspecting my own data between my own endpoints as part of a some 
proving-of-concept, so there's no illegality here, but I appreciate the 
caution.

Using the cache.log and debug provided me with too much data.  With ICAP I'm 
able to apply ACLs to limit the traffic sent to the ICAP server.

Am I right in saying that it is possible to do, I just need the right ICAP 
server?  I'm happy to write one myself, I'm just surprised that it's not been 
done before.  I thought perhaps I was missing an option, say in c-icap or 
some other server.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Bypass squid using iptables

[Amos Jeffries]

On 25/05/20 10:09 pm, Ben Goz wrote:
> B.H
>>Tunneling it elsewhere,
> Where can I tunnel it? and how can I configure my machine to support it?
> 

You will need at least Squid-4, with this line in squid.conf:

  on_unsupported_protocol tunnel

see also 

Squid will blindly tunnel the protocols it does not understand to
whatever server IP:port the client was trying to connect to.


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 4.4 https_port and ssl-bump : Fatal bungled line

[Amos Jeffries]

On 25/05/20 9:59 pm, ben benml wrote:
> Hello,
> 
> I'm contacting you for some help.
> I need to deploy a secure proxy based on Squid.
> 
> I try to use https_port combined with sslbump. I get an error message
> about a bungled line.
> 
> The reasons I want to do this :
> - secure connection between the client browser and the proxy server, so
> using https_port to do it. encrypted  traffic in TLS between the client
> and the server.

Fine. Simply using https_port does that.

> - secure login connection. So I need to use https_port to do this.

Fine. Simply using https_port does that.

> - Do ssl inspection of the traffic goeing through the proxy

Squid does not yet support SSL-Bump decrypt of traffic already being
decrypted for the secure proxy.


Please see
 if
you want details.

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid marking QOS and matching marks with linux iptables problem !

[Amos Jeffries]

On 25/05/20 9:25 pm, Ahmad Alzaeem wrote:
> Here is debug result :
> 
> 
> 
> 2020/05/25 12:04:58.043 kid1| 33,5| client_side.cc
> (1375) parseHttpRequest: Prepare absolute URL from 
> 2020/05/25 12:04:58.043 kid1| 33,5| client_side.cc
> (2106) clientParseRequests:
> local=45.150.17.10:3128 remote=50.254.22.18:62916 FD 540 flags=1: done
> parsing a request

The client connection on FD 540 was open long before this log trace
begins. Any netfilter details fetched are back at the point it was accepted.



> 2020/05/25 12:04:58.043 kid1| 33,3| http/Stream.h(141) mayUseConnection:
> This 0x41e43f0 marked 1

NP: this is a different kind of marking, about whether it is persistent
or not. Not relevant.


...
> 2020/05/25 12:04:58.056 kid1| 17,3| FwdState.cc
> (1339) GetMarkingsToServer: from 45.150.17.10
> netfilter mark 0

This 0 mark is what iptables has set on returning packets for the origin
server connection.

That lien existing at least confirms absolutely that the library and
relevant code is built properly - what Eliezer was looking for with the
squid -v request.


> 2020/05/25 12:04:58.056 kid1| 50,3| comm.cc (350)
> comm_openex: comm_openex: Attempt open socket for: 45.150.17.10
> 2020/05/25 12:04:58.056 kid1| 50,3| comm.cc (393)
> comm_openex: comm_openex: Opened socket local=45.150.17.10 remote=[::]
> FD 542 flags=1 : family=2, type=1, protocol=6

New connection opened, but the log snippet ends before the per-message
socket options are updated for the outgoing HTTP request message.

...



To find the most relevant lines look for "doNfmarkLocalHit",
"doNfmarkLocalMiss" and "setSockNfmark".

If there are errors receiving a MARK from iptables
"getNfmarkFromConnection" will show up too.

When you have found the relevant places, use the FD value on those lines
to grep for more details on what is happening on that connection.


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Bypass squid using iptables

[Ben Goz]

B.H
>Tunneling it elsewhere,
Where can I tunnel it? and how can I configure my machine to support it?

>You cannot have iptables suddenly divert packets to other software
mid-stream.
I want to tunnel it by IP or translate a group of URLs to IPs I'm not sure
if this is the case that you mentioned,
Because I can do it before squid handles TCP session initialization.

The issue here is as I said that I want bypass WSS and other stuff that
squid can't technically support for known list of IPs (or URLS).
Do you have any recommended configuration for this requirement?

Regards,
Ben
suddenly divert packets to other software mid-stream.

‫בתאריך יום ב׳, 25 במאי 2020 ב-9:56 מאת ‪Amos Jeffries‬‏ <‪
squ...@treenet.co.nz‬‏>:‬

> On 21/05/20 3:49 am, Ben Goz wrote:
> > B.H.
> >
> > I'm using squid with c-icap module for specific content filtering. I
> > configured squid with ssl bump so website with WSS won't work on it as
> > mentioned on squid documentation. So for such URLs (with WSS) I need
> > bypassing squid. I read in some posts that squid doesn't fully supports
> > bypassing URLs and best way is to bypasses it via iptables.
> >
> > Eventually I redirects browser traffic to my proxy machine using local
> > machine proxy settings, and Its redirects traffic to my machine with IP
> > x.x.x.x port 3128.
> >
> > If I want to use the conservative iptables bypassing how should I config
> > my machine? and how iptables rules should looks like?
> >
>
> Since you are redirecting the traffic to Squid in the first place. All
> you have to do is *not* redirect the relevant traffic. See your firewall
> software documentation on how to configure that.
>
>
> The hard part is figuring out which traffic you want the proxy to
> service, and what to bypass given only a TCP SYN packet.
>
>
> Be aware that once a TCP SYN+ACK packet is delivered to accept the
> connection Squid *has* to service that TCP connection in its entirety.
> Such 'service' may mean terminating it without any traffic, tunneling it
> elsewhere, or full processing of the traffic.
>  Either way Squid is the agent servicing it. You cannot have iptables
> suddenly divert packets to other software mid-stream.
>
>
> HTH
> Amos
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid cache with SSL

[Amos Jeffries]

On 25/05/20 8:09 pm, Andrey Etush-Koukharenko wrote:
> Hello, I'm trying to set up a cache for GCP signed URLs using squid 4.10
> I've set ssl_bump:
> *http_port 3128 ssl-bump cert=/etc/ssl/squid_ca.pem
> generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
> 
> sslcrtd_program /usr/lib/squid/security_file_certgen -s /var/lib/ssl_db
> -M 4MB
> 
> acl step1 at_step SslBump1
> 
> ssl_bump peek step1
> ssl_bump bump all*

The above SSL-Bump configuration tries to auto-generate server
certificates based only on details in the TLS client handshake. This
leads to a huge number of problems, not least of which is completely
breaking TLS security properties.

Prefer doing the bump at step3,


> *
> I've set cache like this:
> 
> *refresh_pattern -i my-dev.storage.googleapis.com/.* 
> 4320 80% 43200 override-expire ignore-reload ignore-no-store ignore-private*
> *

FYI: that does not setup the cache. It provides *default* parameters for
the heuristic expiry algorithm.

* override-expire replaces the max-age (or Expires header) parameter
with 43200 minutes from object creation.
  This often has the effect of forcing objects to expire from cache long
before they normally would.

* ignore-reload makes Squid ignore requests from the client to update
its cached content.
 This forces content which is stale, outdated, corrupt, or plain wrong
to remain in cache no matter how many times clients try to re-fetch for
a valid response.

* ignore-private makes Squid content that is never supposed to be shared
between clients.
 To prevent personal data being shared between clients who should never
see it Squid will revalidate these objects. Usually different data will
return, making this just a waste of cache space.

* ignore-no-store makes Squid cache objects that are explicitly
*forbidden* to be stored in a cache.
  80% of 0 seconds == 0 seconds before these objects become stale and
expire from cache.

Given that you described this as a problem with an API doing *signing*
of things I expect that at least some of those objects will be security
keys. Possibly generated specifically per-item keys, where forced
caching is a *BAD* idea.

I recommend removing that line entirely from your config file and
letting the Google developers instructions do what they are intended to
do with the cacheability. At the very least start from the default
caching behaviour and see how it works normally before adding protocol
violations and unusual (mis)behvaviours to how the proxy caches things.


> *
> In the cache directory, I see that object was stored after the first
> call, but when I try to re-run the URL I get always
> get: *TCP_REFRESH_UNMODIFIED_ABORTED/200*

What makes you think anything is going wrong?

 Squid found the object in cache (HIT).
 The object requirements were to check with the origin server about
whether it could still be used (HIT becomes REFRESH).
 The origin server said it was fine to deliver (UNMODIFIED).
 Squid started delivery (status 200).
 The client disconnected before the response could be completed delivery
(ABORTED).

Clients are allowed to disconnect at any time, for any reason.


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Squid 4.4 https_port and ssl-bump : Fatal bungled line

[ben benml]

Hello,

I'm contacting you for some help.
I need to deploy a secure proxy based on Squid.

I try to use https_port combined with sslbump. I get an error message about
a bungled line.

The reasons I want to do this :
- secure connection between the client browser and the proxy server, so
using https_port to do it. encrypted  traffic in TLS between the client and
the server.
- secure login connection. So I need to use https_port to do this.
Otherwise If I use http_port, the login/password can be read on the network.
- Do ssl inspection of the traffic goeing through the proxy


What I have done with success :
- https_port without sslbump  (traffic between the brownser and the client
is encrypted. The login/password can't be read on the network)
- ssl-bump on http_port. The ssl inspection is working  ... but the
connexion between the browser and the proxy service is not encrypted

But can't get  'https_port 3129 ssl-bump' working.
FATAL: ssl-bump on https_port requires tproxy/intercept which is missing.
FATAL: Bungled squid.conf line 49: https_port 3129 ssl-bump
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
tls-cert=/etc/squid/squid-cert.pem options=SINGLE_DH_USE,SINGLE_ECDH_USE
tls-dh=/etc/squid/dhparam.pem

Is there something  I have misunderstood ? Or doing wrong ?

I have generated the certificate and CA with openssl :
* openssl req -new -newkey rsa:2048 -days 365 -nodes -x509 -keyout
squid-cert.pem -out squid-cert.pem
* openssl x509 -in squid-cert.pem -outform DER -out squid-CA-browser.der
* openssl dhparam -outform PEM -out dhparam.pem 2048

Squid version : 4.4 from EPEL on centos 8 with  '--enable-ssl'
'--enable-ssl-crtd' '--with-openssl'

Squid configuration as follow :
===
auth_param basic program /usr/lib64/squid/basic_ncsa_auth
/etc/squid/htpasswd
auth_param basic children 10
auth_param basic realm Squid proxy-caching web server
auth_param basic credentialsttl 2 hours

acl localnet src 0.0.0.1-0.255.255.255
acl localnet src 192.168.0.0/24

acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 443
acl CONNECT method CONNECT

http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

#squid mgmt interface access
http_access allow localhost manager
http_access deny manager

acl auth_users proxy_auth REQUIRED
http_access allow auth_users

http_access allow localnet
http_access allow localhost

#squid mgmt interface access
http_access allow localhost manager
http_access deny manager

#http_access deny to_localhost
http_access deny all

##Many Tests here :
#http_port 3128 ssl-bump cert=/etc/squid/squid-cert.pem
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
#http_port 3128 ssl-bump tls-cert=/etc/squid/squid-cert.pem
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB

#http_port 3128 ssl-bump cert=/etc/squid/squid-cert.pem
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
cipher=HIGH:MEDIUM:!LOW:!RC4:!SEED:!IDEA:!3DES:!MD5:!EXP:!PSK:!DSS
options=NO_TLSv1,NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE

#https_port 3129 cert=/etc/squid/squid-cert.pem
#https_port 3129 tls-cert=/etc/squid/squid-cert.pem

https_port 3129 ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=4MB tls-cert=/etc/squid/squid-cert.pem
options=SINGLE_DH_USE,SINGLE_ECDH_USE tls-dh=/etc/squid/dhparam.pem

sslcrtd_program /usr/lib64/squid/security_file_certgen

acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump bump all



tls_outgoing_options min-version=1.0
cipher=HIGH:MEDIUM:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS

#LOGS : deux options. Envoie des logs directe
access_log daemon:/var/log/squid/access.log squid
#access_log tcp://[ip]:[port] squid
access_log syslog:local0.info squid
cache_log /var/log/squid/cache.log rotate=10

#Cache
cache_mem 512 MB
cache_dir ufs /var/spool/squid 1 16 256
coredump_dir /var/spool/squid
===

Thank you in advance !

Regards,
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] squid configuration with c-icap

[Amos Jeffries]

On 25/05/20 7:14 pm, Amiq Nahas wrote:
> Hi Guys,
> 
> At this point, I have got squid installed on my system. I think it is
> working fine since I can browse the internet by adding a manual proxy
> in firefox at localhost:3128.
> 
> What I want now is to configure squid such that it passes the request
> to c-icap. Something like mentioned in this image
> https://postimg.cc/qgsfRbWc
> 
> To elaborate, I want to run a squid proxy such that every time the
> browser or any other application on a system makes request to a url,
> the squid proxy receives it. Then, squid sends it to the c-icap
> server. On the c-icap server there would be a custom module (C
> program) which takes the url as the input and the custom module will
> decide whether the url should be allowed or not.

From that API description there is no need for ICAP which is a very
processing-expensive system.

A simple external ACL helper can perform exactly what you describe far
more efficiently.

Like so:

 external_acl_type urlChecker %>ru /path/to/helper
 acl urlCheck external urlChecker
 http_access deny !urlCheck


Useful documentation on the protocol the helper needs to communicate
with Squid can be found at:
 


> 
> https://wiki.squid-cache.org/ConfigExamples/ContentAdaptation/C-ICAP
> The link above shows the configuration using squidclamav. I do not
> want to use that.

Squid is just an ICAP client. What C-ICAP uses internally is irrelevant
to Squid. squidclamav is just a module people tend to ask for, so the
example shows it. The squid.conf rules will be the same or similar for
whatever your system has (assuming you stay with ICAP instead of moving
to external ACL for access control).

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid marking QOS and matching marks with linux iptables problem !

[Ahmad Alzaeem]

Here is debug result :



2020/05/25 12:04:58.043 kid1| 33,5| client_side.cc(1375) parseHttpRequest: 
Prepare absolute URL from 
2020/05/25 12:04:58.043 kid1| 33,5| client_side.cc(2106) clientParseRequests: 
local=45.150.17.10:3128 remote=50.254.22.18:62916 FD 540 flags=1: done parsing 
a request
2020/05/25 12:04:58.043 kid1| 33,3| Pipeline.cc(24) add: Pipeline 0x43d98a0 add 
request 1 0x41e43f0*4
2020/05/25 12:04:58.043 kid1| 33,5| Http1Server.cc(188) buildHttpRequest: 
normalize 1 Host header using analytics.yopify.com:443
2020/05/25 12:04:58.043 kid1| 33,3| client_side.cc(641) clientSetKeepaliveFlag: 
http_ver = HTTP/1.1
2020/05/25 12:04:58.043 kid1| 33,3| client_side.cc(642) clientSetKeepaliveFlag: 
method = CONNECT
2020/05/25 12:04:58.043 kid1| 33,3| http/Stream.h(141) mayUseConnection: This 
0x41e43f0 marked 1
2020/05/25 12:04:58.043 kid1| 50,3| comm.cc(946) comm_udp_sendto: 
comm_udp_sendto: Attempt to send UDP packet to 8.8.8.8:53 using FD 8 using Port 
55332
2020/05/25 12:04:58.043 kid1| 50,3| comm.cc(946) comm_udp_sendto: 
comm_udp_sendto: Attempt to send UDP packet to 8.8.8.8:53 using FD 8 using Port 
55332
2020/05/25 12:04:58.043 kid1| 33,3| client_side.cc(2119) clientParseRequests: 
Not parsing new requests, as this request may need the connection
2020/05/25 12:04:58.044 kid1| 33,5| AsyncJob.cc(154) callEnd: Http1::Server 
status out: [ job690]
2020/05/25 12:04:58.044 kid1| 33,5| AsyncCallQueue.cc(57) fireNext: leaving 
Server::doClientRead(local=45.150.17.10:3128 remote=50.254.22.18:62916 FD 540 
flags=1, data=0x43d9858)
2020/05/25 12:04:58.056 kid1| 17,3| FwdState.cc(1339) GetMarkingsToServer: from 
45.150.17.10 netfilter mark 0
2020/05/25 12:04:58.056 kid1| 50,3| comm.cc(350) comm_openex: comm_openex: 
Attempt open socket for: 45.150.17.10
2020/05/25 12:04:58.056 kid1| 50,3| comm.cc(393) comm_openex: comm_openex: 
Opened socket local=45.150.17.10 remote=[::] FD 542 flags=1 : family=2, type=1, 
protocol=6
2020/05/25 12:04:58.064 kid1| 33,4| client_side.cc(2510) httpAccept: 
local=45.150.17.10:3128 remote=50.254.22.18:62917 FD 543 flags=1: accepted
2020/05/25 12:04:58.064 kid1| 33,5| AsyncCall.cc(26) AsyncCall: The AsyncCall 
ConnStateData::connStateClosed constructed, this=0x4024ec0 [call6687]
2020/05/25 12:04:58.064 kid1| 33,5| AsyncCall.cc(26) AsyncCall: The AsyncCall 
Http1::Server::requestTimeout constructed, this=0x422ab40 [call6688]
2020/05/25 12:04:58.064 kid1| 33,4| Server.cc(90) readSomeData: 
local=45.150.17.10:3128 remote=50.254.22.18:62917 FD 543 flags=1: reading 
request...
2020/05/25 12:04:58.064 kid1| 33,5| AsyncCall.cc(26) AsyncCall: The AsyncCall 
Server::doClientRead constructed, this=0x4025c50 [call6689]



I see mark 0 and mark 1 , Dont see any 0xd7 or so .

Thanks 

> On May 25, 2020, at 10:02 AM, Amos Jeffries  wrote:
> 
> [NP: it would help if you replied through the list instead of directly
> to me, even as a CC. Your messages keep getting diverted to spam folder. ]
> 
> On 25/05/20 4:26 am, Ahmad Alzaeem wrote:
>> Hi Amos , 
>> 
>> Sorry I'm confused a a bit …
>> 
>> Are my results expected not to work with below :
>> 
>> 
>> qos_flows mark local-hit=0xd7
>> qos_flows mark local-miss=0xd7
>> 
>> 
>> -A OUTPUT -m mark --mark 0xd7 -j ACCEPT
>> -A OUTPUT -m connmark --mark 0xd7 -j ACCEPT
>> 
>> ?
> 
> Squid should be MARK'ing packets with 0xd7.
> 
> Those iptables rules should match the packets MARK'ed with 0xd7.
> 
> Whether those statements are of any relevance depends on where your
> iptables rules are configured in relation to all other rules and chains
> your iptables is processing.
> 
> 
>> 
>> Do I need to edit squid/iptables ?
>> 
> 
> Probably iptables. But not enough info to say how.
> 
> 
> You asked about how to debug Squid MARK'ing earlier. What were the
> results of that? did you see Squid doing any marking?
> 
> 
> Amos
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org 
> http://lists.squid-cache.org/listinfo/squid-users 
> 
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Squid cache with SSL

[Andrey Etush-Koukharenko]

Hello, I'm trying to set up a cache for GCP signed URLs using squid 4.10
I've set ssl_bump:







*http_port 3128 ssl-bump cert=/etc/ssl/squid_ca.pem
generate-host-certificates=on
dynamic_cert_mem_cache_size=4MBsslcrtd_program
/usr/lib/squid/security_file_certgen -s /var/lib/ssl_db -M 4MBacl step1
at_step SslBump1ssl_bump peek step1ssl_bump bump all*

I've set cache like this:

*refresh_pattern -i my-dev.storage.googleapis.com/.*

4320 80% 43200 override-expire ignore-reload ignore-no-store ignore-private*

In the cache directory, I see that object was stored after the first call,
but when I try to re-run the URL I get always get:
*TCP_REFRESH_UNMODIFIED_ABORTED/200*

and I get the empty object, I've tried to play with *refresh_pattern *params
but still no luck.

Thanks for your help
Andrey
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] squid configuration with c-icap

[Amiq Nahas]

Hi Guys,

At this point, I have got squid installed on my system. I think it is
working fine since I can browse the internet by adding a manual proxy
in firefox at localhost:3128.

What I want now is to configure squid such that it passes the request
to c-icap. Something like mentioned in this image
https://postimg.cc/qgsfRbWc

To elaborate, I want to run a squid proxy such that every time the
browser or any other application on a system makes request to a url,
the squid proxy receives it. Then, squid sends it to the c-icap
server. On the c-icap server there would be a custom module (C
program) which takes the url as the input and the custom module will
decide whether the url should be allowed or not.

https://wiki.squid-cache.org/ConfigExamples/ContentAdaptation/C-ICAP
The link above shows the configuration using squidclamav. I do not
want to use that. I just want squid proxy to send requested urls to
c-icap server after that the c-icap server will decide whether to
block or allow the url. I will handle the block and allow part later.
For now I would like to know about how to configure squid such that
all requests from my system are first given to squid proxy which
further passes it to c-icap.

Thanks
Amiq
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid marking QOS and matching marks with linux iptables problem !

[Amos Jeffries]

[NP: it would help if you replied through the list instead of directly
to me, even as a CC. Your messages keep getting diverted to spam folder. ]

On 25/05/20 4:26 am, Ahmad Alzaeem wrote:
> Hi Amos , 
> 
> Sorry I'm confused a a bit …
> 
> Are my results expected not to work with below :
> 
> 
> qos_flows mark local-hit=0xd7
> qos_flows mark local-miss=0xd7
> 
> 
> -A OUTPUT -m mark --mark 0xd7 -j ACCEPT
> -A OUTPUT -m connmark --mark 0xd7 -j ACCEPT
> 
> ?

Squid should be MARK'ing packets with 0xd7.

Those iptables rules should match the packets MARK'ed with 0xd7.

Whether those statements are of any relevance depends on where your
iptables rules are configured in relation to all other rules and chains
your iptables is processing.


> 
> Do I need to edit squid/iptables ?
> 

Probably iptables. But not enough info to say how.


You asked about how to debug Squid MARK'ing earlier. What were the
results of that? did you see Squid doing any marking?


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Bypass squid using iptables

[Amos Jeffries]

On 21/05/20 3:49 am, Ben Goz wrote:
> B.H.
> 
> I'm using squid with c-icap module for specific content filtering. I
> configured squid with ssl bump so website with WSS won't work on it as
> mentioned on squid documentation. So for such URLs (with WSS) I need
> bypassing squid. I read in some posts that squid doesn't fully supports
> bypassing URLs and best way is to bypasses it via iptables.
> 
> Eventually I redirects browser traffic to my proxy machine using local
> machine proxy settings, and Its redirects traffic to my machine with IP
> x.x.x.x port 3128.
> 
> If I want to use the conservative iptables bypassing how should I config
> my machine? and how iptables rules should looks like?
> 

Since you are redirecting the traffic to Squid in the first place. All
you have to do is *not* redirect the relevant traffic. See your firewall
software documentation on how to configure that.


The hard part is figuring out which traffic you want the proxy to
service, and what to bypass given only a TCP SYN packet.


Be aware that once a TCP SYN+ACK packet is delivered to accept the
connection Squid *has* to service that TCP connection in its entirety.
Such 'service' may mean terminating it without any traffic, tunneling it
elsewhere, or full processing of the traffic.
 Either way Squid is the agent servicing it. You cannot have iptables
suddenly divert packets to other software mid-stream.


HTH
Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid does not cache file download by FileZilla and apache FTPCLIENT

[Amos Jeffries]

On 25/05/20 6:06 pm, david770514 wrote:
> Hi Amos,
> 
> The "apache.commons.net.ftp.FTPHTTPClient" is sent as CONNECT tunnels
> through the proxy. Can I make it work through modifying the Squid? Let Squid
> can cache file when I sent as CONNECT tunnels through the proxy?
> 

Since it uses the tunnel mechanism. No you cannot cache it.

What is inside the tunnel is not just a "file" downloaded. It is a whole
set of FTP messages going back and forth negotiating how the client is
logging into the server (anonymous or not), moving working directory
around within the FTP server and files (possibly more than one)
accessed, metadata about the files, and any temporary TCP connection
details of other tunnels being used for additional pieces of the
communication.

There is simply no way all that stuff can be cached and replayed as-is
to any other client without serious breakage happening.

Sorry, but unless you can find a way to get the client(s) to send Squid
ftp:// URLs requests in HTTP messages there is no caching.

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Dumping sslbump'd decrytped http using icap protocol

[Amos Jeffries]

On 25/05/20 12:56 am, Scott wrote:
> Hi,
> 
> Can someone recommend an ICAP application that will allow me to dump the HTTP 
> of a client-server conversation?
> 
> I am doing some forensics on an app - I have sslbump configured correctly and 
> I can get the traffic to c-icap (for example).
> 
> I'd like to dump this to a text file.
> 
> Is there a dump option for c-icap?  I couldn't find one.
> 

FYI; this action is illegal in a lot of places. Even answering your
question can be quite risky.


To perform traffic forensics you can use the Squid cache.log directly
and not involve any insecure third-party software or communication
dumps. See 
for more details.

"debug_Options 11,2" is probably all you need.


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Docker squid container setup

[Amos Jeffries]

On 25/05/20 4:29 am, pmohan wrote:
> how do you set siblings in a docker swam setup .. squid config has to be
> different isnt it ?

That depends on how the containers are configured. Modern Squid use mDNS
or regular DNS name lookup for cache_peer.

So long as the containers have different IPs each, the (m)DNS can
provide those to hostname looks, and your network setup allows them to
communicate there should be no difference between a bunch of VMs,
containers, hardware devices, or any combo you want.


With squid.conf using ${process_name} you could also mix in multi-tenant
instead of containerizing the Squid.


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid does not cache file download by FileZilla and apache FTPCLIENT

[david770514]

Hi Amos,

The "apache.commons.net.ftp.FTPHTTPClient" is sent as CONNECT tunnels
through the proxy. Can I make it work through modifying the Squid? Let Squid
can cache file when I sent as CONNECT tunnels through the proxy?



--
Sent from: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users