[squid-users] Call for adaptation after sni peeked

[Jatin Bhasin]

Hi All,

This question is related to ssl decryption and ecap adaptation call.
When the ssl connection starts then before it even extracts sni squid sends
 fakeConnect which comes to ecap as well.
I am using peek in step 1 and after fakeConnect squid extracts the sni, but
at this point squid does not make another call to ecap. This function in
squid is startPeekAndSpliceDone in file client_side.cc
In this function it only makes a call to acl for ssl bump to check but no
call to ecap adaptation checks.

I was hoping at this point I can put a call to http->doCallouts which can
make the call to ecap adapter and this time we have sni as well?

I needed this functionality as I want to make the decision using sni
whether to bump the connection or not.

Thanks,
Jatin Bhasin
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] assertion failed: Controller.cc:838: "!transients || e.hasTransients()"

[Alex Rousskov]

On 10/22/19 3:24 PM, Antonio SJ Musumeci wrote:
> Squid 4.8
> 
> Attempting to get a SMP setup with rock enabled. Instance points to an
> origin web server.
> 
> With "workers 1" everything appears to work fine. If I set "workers 2" I
> get a number of issues:
> 
> If I request a particular object a kid will fail with:

> assertion failed: Controller.cc:838: "!transients || e.hasTransients()"

For supported Squid configurations with no other reported errors or
warnings, this assertion indicates a Squid bug. Consider filing a bug
report and posting a stack trace (as well as other relevant reproduction
info) there. Posting an ALL,9 cache.log collected while executing a
single asserting transaction may speed up triage.


> Is there a simple, modern, functional SMP example to compare?

To enable basic SMP features, adding "cache_dir rock..." or "workers N"
(with N greater than 1) is enough.

Alex.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] assertion failed: Controller.cc:838: "!transients || e.hasTransients()"

[Antonio SJ Musumeci]

Squid 4.8

Attempting to get a SMP setup with rock enabled. Instance points to an 
origin web server.


With "workers 1" everything appears to work fine. If I set "workers 2" I 
get a number of issues:


If I request a particular object a kid will fail with:

assertion failed: Controller.cc:838: "!transients || e.hasTransients()"

Pulling other objects sometimes works but speeds continuously decline to 
the point of being unusable. I'm not seeing any consistency between what 
objects cause what issues and any acls or refresh patterns.


Is there a simple, modern, functional SMP example to compare? Does that 
assert stand out? Didn't find anything on search engines.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] (no subject)

[Vieri Di Paola]

On Tue, Oct 22, 2019 at 1:48 PM Amos Jeffries  wrote:
>
> I do not see any DIVERT rule at all in your firewall config dump. That
> is at least part of the problem.

I opened the previous dump and saw the divert rules here below:

Chain PREROUTING (policy ACCEPT 573K packets, 462M bytes)
 pkts bytes target prot opt in out source
destination
 573K  462M CONNMARK   all  --  *  *   0.0.0.0/0
0.0.0.0/0CONNMARK restore mask 0xff
 1213  181K routemark  all  --  ppp1   *   0.0.0.0/0
0.0.0.0/0mark match 0x0/0xff
 3195  308K routemark  all  --  ppp2   *   0.0.0.0/0
0.0.0.0/0mark match 0x0/0xff
 1320 79360 routemark  all  --  ppp3   *   0.0.0.0/0
0.0.0.0/0mark match 0x0/0xff
 311K  277M tcpre  all  --  *  *   0.0.0.0/0
0.0.0.0/0mark match 0x0/0xff
0 0 divert tcp  --  ppp1   *   0.0.0.0/0
10.215.144.48   [goto]  tcp spt:80 flags:!0x17/0x02 socket
--transparent
0 0 divert tcp  --  ppp2   *   0.0.0.0/0
10.215.144.48   [goto]  tcp spt:80 flags:!0x17/0x02 socket
--transparent
0 0 divert tcp  --  ppp3   *   0.0.0.0/0
10.215.144.48   [goto]  tcp spt:80 flags:!0x17/0x02 socket
--transparent
   76  7484 TPROXY tcp  --  enp10s0 *   10.215.144.48
0.0.0.0/0tcp dpt:80 TPROXY redirect 0.0.0.0:3129 mark
0x200/0x200
0 0 divert tcp  --  ppp1   *   0.0.0.0/0
10.215.144.48   [goto]  tcp spt:443 flags:!0x17/0x02 socket
--transparent
0 0 divert tcp  --  ppp2   *   0.0.0.0/0
10.215.144.48   [goto]  tcp spt:443 flags:!0x17/0x02 socket
--transparent
0 0 divert tcp  --  ppp3   *   0.0.0.0/0
10.215.144.48   [goto]  tcp spt:443 flags:!0x17/0x02 socket
--transparent
   10  1060 TPROXY tcp  --  enp10s0 *   10.215.144.48
0.0.0.0/0tcp dpt:443 TPROXY redirect 0.0.0.0:3130 mark
0x200/0x200

Aren't these the DIVERT rules you are referring to?
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] (no subject)

[Vieri Di Paola]

On Tue, Oct 22, 2019 at 1:48 PM Amos Jeffries  wrote:
>
> On 22/10/19 11:22 pm, Vieri Di Paola wrote:
> >
> > I use Shorewall on this system. This program configures iptables and 
> > routing.
> > I dumped all the network information while trying to access port 80 on
> > host with IP addr. 104.113.250.104 form local host with IP addr.
> > 10.215.144.48:
> I do not see any DIVERT rule at all in your firewall config dump. That
> is at least part of the problem.

I don't know why.. I must have taken the wrong dump. Here's a new one
I just tested:

https://drive.google.com/file/d/1iqIU8SrvmOfSHs7wv2tjLLx1DXWNrP8h/view?usp=sharing

> Have you run through the notes and troubleshooting checks on the TPROXY
> feature page?
> 

Yes, but I'm obviously overlooking something.
I'll work on it.

Thanks,

Vieri
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] (no subject)

[Amos Jeffries]

On 22/10/19 11:22 pm, Vieri Di Paola wrote:
> 
> I use Shorewall on this system. This program configures iptables and routing.
> I dumped all the network information while trying to access port 80 on
> host with IP addr. 104.113.250.104 form local host with IP addr.
> 10.215.144.48:


I do not see any DIVERT rule at all in your firewall config dump. That
is at least part of the problem.

Have you run through the notes and troubleshooting checks on the TPROXY
feature page?



Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] (no subject)

[Vieri Di Paola]

Hi,

On Fri, Oct 18, 2019 at 10:13 PM Amos Jeffries  wrote:
>
> If you are able to share your config maybe we could help spot something,
> both for that and for the timeout issue.

I prepared and tested a trimmed-down squid conf:

# cat squid.conf
acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 901 # SWAT
acl CONNECT method CONNECT

http_access deny !Safe_ports

http_access deny CONNECT !SSL_ports

http_access allow localhost manager
http_access deny manager

acl explicit myportname 3128
acl intercepted myportname 3129
acl interceptedssl myportname 3130

http_port 3128
http_port 3129 tproxy
https_port 3130 tproxy ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=16MB cert=/etc/ssl/squid/proxyserver.pem
sslflags=NO_DEFAULT_CA
sslproxy_flags DONT_VERIFY_PEER

sslcrtd_program /usr/libexec/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 16MB
sslcrtd_children 40 startup=20 idle=10

cache_dir diskd /var/cache/squid 32 16 256

acl localnet src 10.0.0.0/8
acl localnet src 192.168.0.0/16

acl good_useragents req_header User-Agent Firefox/
acl good_useragents req_header User-Agent Edge/
acl good_useragents req_header User-Agent Microsoft-CryptoAPI/

http_access deny intercepted !localnet
http_access deny interceptedssl !localnet

http_access allow CONNECT interceptedssl SSL_ports
http_access deny !good_useragents

http_access allow localnet

debug_options rotate=1 ALL,9

reply_header_access Alternate-Protocol deny all
ssl_bump stare all
ssl_bump bump all

icap_enable on
icap_send_client_ip on
icap_send_client_username on
icap_client_username_encode off
icap_client_username_header X-Authenticated-User
icap_preview_enable on
icap_preview_size 1024
icap_service antivirus respmod_precache bypass=0 icap://127.0.0.1:1344/clamav
adaptation_access antivirus allow all

email_err_data on
client_lifetime 480 minutes

httpd_suppress_version_string on
dns_v4_first on
via off
forwarded_for transparent

cache_mem 32 MB

max_filedescriptors 65536
icap_service_failure_limit -1
icap_persistent_connections off

http_access allow localhost

http_access deny all

coredump_dir /var/cache/squid

refresh_pattern ^ftp:   144020% 10080
refresh_pattern ^gopher:14400%  1440
refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
refresh_pattern .   0   20% 4320

> You said Squid used TPROXY. The spoofing of packets causes a different
> set of routing tables and rules to be applied than normal server
> outgoing traffic.

I use Shorewall on this system. This program configures iptables and routing.
I dumped all the network information while trying to access port 80 on
host with IP addr. 104.113.250.104 form local host with IP addr.
10.215.144.48:
https://drive.google.com/file/d/13Pr2OCgCInY6E72krCci9BiHrB1lrMce/view?usp=sharing

> Looks like Squid is doing everything right and the issues is somewhere
> between the TCP SYN send and SYN ACK returning.

I suspect there must be something wrong with my routing or marking
(please see dump).

Thanks,

Vieri
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] external_acl_type and ipv6

[Amos Jeffries]

On 22/10/19 7:31 pm, Vieri Di Paola wrote:
> Hi,
> 
> What is the advantage of using ipv6 instead of ipv4 by default for
> external_acl_type?
> 

* larger packet sizes,

* simplicity, and

* all modern OS support IPv6 by default.

There is also:



"
   o  IPv6 support must be equivalent or better in quality and
  functionality when compared to IPv4 support in a new or updated IP
  implementation.

   o  New and updated IP networking implementations should support IPv4
  and IPv6 coexistence (dual-stack), but must not require IPv4 for
  proper and complete function.

   o  Implementers are encouraged to update existing hardware and
  software to enable IPv6 wherever technically feasible.
"

No reason not to comply on private channels created between Squid and
its helpers.

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users