date:20220810

Re: [squid-users] Problem caching range requests with range_offset_limit

2022-08-10 Thread Alex Rousskov


On 8/10/22 22:43, osy wrote:

I am trying to cache Azure objects. I have SSL bump and the following config:

maximum_object_size 1000 MB
acl azure_storage dstdomain .blob.core.windows.net
range_offset_limit -1 azure_storage

My expectation is that when a request is made with the Range header,
that Squid will request the object starting from the beginning, return
the data when it reaches the range, then keep the connection open and
keep requesting data until the end.


N.B. In the above "keep the connection" should refer to the 
Squid-to-origin connection, not the user_agent-to-Squid connection.




However, I observe the last part
isn't true: the connection to the server is closed after the proxy
client closes the connection.

2022/08/10 18:55:23.152| 5,3| Read.cc(92) ReadNow:
local=127.0.0.1:3128 remote=127.0.0.1:57075 FD 21 flags=1, size 3644,
retval 0, errno 0
2022/08/10 18:55:23.152| 33,5| Server.cc(147) doClientRead:
local=127.0.0.1:3128 remote=127.0.0.1:57075 FD 21 flags=1 closed?
2022/08/10 18:55:23.152| 33,3| client_side.cc(1418)
connFinishedWithConn: local=127.0.0.1:3128 remote=127.0.0.1:57075 FD
21 flags=1 aborted (half_closed_clients disabled)
2022/08/10 18:55:23.152| 33,3| Pipeline.cc(56) terminateAll: Pipeline
0x149704e80 notify(0) 0x13a817600*3
2022/08/10 18:55:23.152| 90,3| store_client.cc(651) storeUnregister:
storeUnregister: called for 'F43F6204305C4808C2A393C7CC905C16'
2022/08/10 18:55:23.153| 90,3| store_client.cc(764)
CheckQuickAbortIsReasonable: entry=e:d5@0=w1p2V/0x638f6d80*6
2022/08/10 18:55:23.153| 90,3| store_client.cc(755)
storePendingNClients: storePendingNClients: returning 0
2022/08/10 18:55:23.153| 90,3| store_client.cc(788)
CheckQuickAbortIsReasonable: mem=0x14a14bcb0
2022/08/10 18:55:23.153| 90,3| store_client.cc(817)
CheckQuickAbortIsReasonable: quick-abort? NO admin configured range
replies to full-download
2022/08/10 18:55:23.153| 33,3| Pipeline.cc(69) popMe: Pipeline
0x149704e80 drop 0x13a817600*3
2022/08/10 18:55:23.153| 33,3| client_side_request.cc(270)
~ClientHttpRequest: httpRequestFree: [url here]


Just to avoid any misunderstanding, the above debugging quote does _not_ 
show the connection to the origin server being closed. There are many 
things called "servers" and "clients" in this context. The "Server.cc" 
above is a Squid task parsing requests from user agents and responding 
with a replies. It is nearly unrelated to the origin server.


The above trace does not necessarily contradict your configuration and 
expectations AFAICT. It shows that the user agent closed the connection 
to Squid, but Squid may have decided to keep going with downloading the 
response from the origin server ("quick-abort? NO"). Something else 
could have gone wrong with that download later (or even earlier), of 
course, but the trace does not show that AFAICT.



HTH,

Alex.




I also tried setting "half_closed_clients on" but that just defers the
error to a write later on. Any advice would be appreciated, thanks!
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Problem caching range requests with range_offset_limit

2022-08-10 Thread osy

I am trying to cache Azure objects. I have SSL bump and the following config:

maximum_object_size 1000 MB
acl azure_storage dstdomain .blob.core.windows.net
range_offset_limit -1 azure_storage

My expectation is that when a request is made with the Range header,
that Squid will request the object starting from the beginning, return
the data when it reaches the range, then keep the connection open and
keep requesting data until the end. However, I observe the last part
isn't true: the connection to the server is closed after the proxy
client closes the connection.

2022/08/10 18:55:23.152| 5,3| Read.cc(92) ReadNow:
local=127.0.0.1:3128 remote=127.0.0.1:57075 FD 21 flags=1, size 3644,
retval 0, errno 0
2022/08/10 18:55:23.152| 33,5| Server.cc(147) doClientRead:
local=127.0.0.1:3128 remote=127.0.0.1:57075 FD 21 flags=1 closed?
2022/08/10 18:55:23.152| 33,3| client_side.cc(1418)
connFinishedWithConn: local=127.0.0.1:3128 remote=127.0.0.1:57075 FD
21 flags=1 aborted (half_closed_clients disabled)
2022/08/10 18:55:23.152| 33,3| Pipeline.cc(56) terminateAll: Pipeline
0x149704e80 notify(0) 0x13a817600*3
2022/08/10 18:55:23.152| 90,3| store_client.cc(651) storeUnregister:
storeUnregister: called for 'F43F6204305C4808C2A393C7CC905C16'
2022/08/10 18:55:23.153| 90,3| store_client.cc(764)
CheckQuickAbortIsReasonable: entry=e:d5@0=w1p2V/0x638f6d80*6
2022/08/10 18:55:23.153| 90,3| store_client.cc(755)
storePendingNClients: storePendingNClients: returning 0
2022/08/10 18:55:23.153| 90,3| store_client.cc(788)
CheckQuickAbortIsReasonable: mem=0x14a14bcb0
2022/08/10 18:55:23.153| 90,3| store_client.cc(817)
CheckQuickAbortIsReasonable: quick-abort? NO admin configured range
replies to full-download
2022/08/10 18:55:23.153| 33,3| Pipeline.cc(69) popMe: Pipeline
0x149704e80 drop 0x13a817600*3
2022/08/10 18:55:23.153| 33,3| client_side_request.cc(270)
~ClientHttpRequest: httpRequestFree: [url here]

I also tried setting "half_closed_clients on" but that just defers the
error to a write later on. Any advice would be appreciated, thanks!
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] SQL DB squid.conf backend, who was it that asked about it?

2022-08-10 Thread marcelorodrigo


Hi Amos,

It was me indeed.
We have developed a squid based php application to create VPSs and 
deliver proxies via web panel.
It is still in development, but fase 1 is working already running SQL 
user management, create VPSs and squid.conf auto configuration.
We are heading to fase 2 to use cache pears and IPv4/IPv6 routing 
depends on source.


Squid.conf got so complex at this point that its getting very hard to 
implement fase 2.


Lack of deep squid knowledge is still our weak spot.

Thanks for asking.

Marcelo.


On 2022-08-08 09:00, squid-users-requ...@lists.squid-cache.org wrote:

Send squid-users mailing list submissions to
squid-users@lists.squid-cache.org

To subscribe or unsubscribe via the World Wide Web, visit
http://lists.squid-cache.org/listinfo/squid-users
or, via email, send a message with subject or body 'help' to
squid-users-requ...@lists.squid-cache.org

You can reach the person managing the list at
squid-users-ow...@lists.squid-cache.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of squid-users digest..."


Today's Topics:

   1. Re: SQL DB squid.conf backend, who was it that asked about
  it? (Amos Jeffries)


--

Message: 1
Date: Mon, 8 Aug 2022 22:55:43 +1200
From: Amos Jeffries 
To: squid-users@lists.squid-cache.org
Subject: Re: [squid-users] SQL DB squid.conf backend, who was it that
asked about it?
Message-ID: <20bf98c6-6a22-f303-928e-29bffbf9f...@treenet.co.nz>
Content-Type: text/plain; charset=UTF-8; format=flowed

On 8/08/22 03:16, ngtech1ltd wrote:

Hey Everybody,

I don?t remember who was it but I was asked about using a SQL DB 
backend

for squid.conf.



FWIW, "Marcello" asked on 10 July. He seems to have moved on to 
external

ACL development (guessing from later thread topics).


Cheers
Amos


--

Subject: Digest Footer

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


--

End of squid-users Digest, Vol 96, Issue 21
***

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 4.8+ intercept

2022-08-10 Thread ngtech1ltd

Hey K,
 
Here a video example on how to implement what you probably want:
https://cloud1.ngtech.co.il/static/squid-data/mikrotik-v7-intercept.mp4
 
If the proxy sits in the same network that the clients sit it won’t work.
 
Eliezer
 

Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email:   ngtech1...@gmail.com
Web:   https://ngtech.co.il/
My-Tube:   https://tube.ngtech.co.il/
 
From: squid-users  On Behalf Of M K
Sent: Tuesday, 9 August 2022 22:29
To: squid-users@lists.squid-cache.org
Subject: [squid-users] Squid 4.8+ intercept
 
Hello,
 
I have a setup like this one:

| Client | => | Router | => Internet
 ||
 \/
  | Squid |
 
...the router is a Mikrotik router capable of all things NAT/Redirect and 
whatnot. Squid server has only one network interface.
Using the router:
- I tried routing traffic to squid server IP.
- I tried destination-NATing from client to server IP, with origin server 
IP-and-port natted to squid IP-and-port, and with origin server IP-only natted 
to squid-IP.
 
I have been struggling for 2 days to setup a working Squid 4.8 or higher 
interception.
Test server is running Ubuntu 18.4.3 and Squid 4.8.
Documentation is either too much trim or extremely outdated.
Any help would be very much appreciated.
 
All best,
K
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 4.8+ intercept

2022-08-10 Thread Rafael Akchurin

Thanks – will do when preparing web safety 8.2 (October/November 2022)  – added 
https://github.com/diladele/websafety/issues/1869

Best regards,
rafael

From: squid-users  On Behalf Of 
ngtech1...@gmail.com
Sent: Wednesday, August 10, 2022 10:10 AM
To: squid-users@lists.squid-cache.org
Subject: Re: [squid-users] Squid 4.8+ intercept

Hey Rafael,

This document covers on the V6 branch of Mikrotik and the stable is 7.4.
If you do have the resources to publish a V7 document upgrade it would help 
others.

Eliezer


Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email: ngtech1...@gmail.com
Web: https://ngtech.co.il/
My-Tube: https://tube.ngtech.co.il/

From: squid-users 
mailto:squid-users-boun...@lists.squid-cache.org>>
 On Behalf Of Rafael Akchurin
Sent: Tuesday, 9 August 2022 23:54
To: M K mailto:mohammed.khal...@gmail.com>>
Cc: squid-users@lists.squid-cache.org
Subject: Re: [squid-users] Squid 4.8+ intercept

Hello K,

We use https://docs.diladele.com/tutorials/mikrotik_transparent_squid/index.html
Best regards,
Rafael


Op 9 aug. 2022 om 21:29 heeft M K 
mailto:mohammed.khal...@gmail.com>> het volgende 
geschreven:

Hello,

I have a setup like this one:

| Client | => | Router | => Internet
 ||
 \/
  | Squid |

...the router is a Mikrotik router capable of all things NAT/Redirect and 
whatnot. Squid server has only one network interface.
Using the router:
- I tried routing traffic to squid server IP.
- I tried destination-NATing from client to server IP, with origin server 
IP-and-port natted to squid IP-and-port, and with origin server IP-only natted 
to squid-IP.

I have been struggling for 2 days to setup a working Squid 4.8 or higher 
interception.
Test server is running Ubuntu 18.4.3 and Squid 4.8.
Documentation is either too much trim or extremely outdated.
Any help would be very much appreciated.

All best,
K
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 4.8+ intercept

2022-08-10 Thread ngtech1ltd

Hey Rafael,
 
This document covers on the V6 branch of Mikrotik and the stable is 7.4.
If you do have the resources to publish a V7 document upgrade it would help 
others.
 
Eliezer
 

Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email: ngtech1...@gmail.com  
Web: https://ngtech.co.il/
My-Tube: https://tube.ngtech.co.il/
 
From: squid-users  On Behalf Of 
Rafael Akchurin
Sent: Tuesday, 9 August 2022 23:54
To: M K 
Cc: squid-users@lists.squid-cache.org
Subject: Re: [squid-users] Squid 4.8+ intercept
 
Hello K, 
 
We use https://docs.diladele.com/tutorials/mikrotik_transparent_squid/index.html
Best regards, 
Rafael



Op 9 aug. 2022 om 21:29 heeft M K mailto:mohammed.khal...@gmail.com> > het volgende geschreven:
 
Hello, 
 
I have a setup like this one:

| Client | => | Router | => Internet
 ||
 \/
  | Squid |
 
...the router is a Mikrotik router capable of all things NAT/Redirect and 
whatnot. Squid server has only one network interface.
Using the router:
- I tried routing traffic to squid server IP.
- I tried destination-NATing from client to server IP, with origin server 
IP-and-port natted to squid IP-and-port, and with origin server IP-only natted 
to squid-IP.
 
I have been struggling for 2 days to setup a working Squid 4.8 or higher 
interception.
Test server is running Ubuntu 18.4.3 and Squid 4.8.
Documentation is either too much trim or extremely outdated.
Any help would be very much appreciated.
 
All best,
K
___
squid-users mailing list
squid-users@lists.squid-cache.org  
http://lists.squid-cache.org/listinfo/squid-users
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 4.8+ intercept

2022-08-10 Thread ngtech1ltd

Hey K,
 
I am not sure about the network topology.
Preferably the Squid should reside on another network then the clients if it’s 
intercepting the traffic.
Also, I assume it’s not a TPROXY setup so it should be pretty simple and 
straight forward.
 
I understand why are you asking this question.
Also take into account that Mikrotik is now on 7.4 firmware and it’s 
recommended to use this one.
If you are using any other version let me know so I can try to make sense on 
the differences.
I will try to give a DEMO for such a setup and how to make it work.
 
Eliezer
 

Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email:   ngtech1...@gmail.com
Web:   https://ngtech.co.il/
My-Tube:   https://tube.ngtech.co.il/
 
From: squid-users  On Behalf Of M K
Sent: Tuesday, 9 August 2022 22:29
To: squid-users@lists.squid-cache.org
Subject: [squid-users] Squid 4.8+ intercept
 
Hello,
 
I have a setup like this one:

| Client | => | Router | => Internet
 ||
 \/
  | Squid |
 
...the router is a Mikrotik router capable of all things NAT/Redirect and 
whatnot. Squid server has only one network interface.
Using the router:
- I tried routing traffic to squid server IP.
- I tried destination-NATing from client to server IP, with origin server 
IP-and-port natted to squid IP-and-port, and with origin server IP-only natted 
to squid-IP.
 
I have been struggling for 2 days to setup a working Squid 4.8 or higher 
interception.
Test server is running Ubuntu 18.4.3 and Squid 4.8.
Documentation is either too much trim or extremely outdated.
Any help would be very much appreciated.
 
All best,
K
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 4.8+ intercept

2022-08-10 Thread Matus UHLAR - fantomas

On 09.08.22 21:28, M K wrote:

I have a setup like this one:

| Client | => | Router | => Internet
||
\/
| Squid |

...the router is a Mikrotik router capable of all things NAT/Redirect and
whatnot. Squid server has only one network interface.
Using the router:
- I tried routing traffic to squid server IP.
- I tried destination-NATing from client to server IP, with origin server
IP-and-port natted to squid IP-and-port, and with origin server IP-only
natted to squid-IP.

https://wiki.squid-cache.org/SquidFaq/InterceptionProxy#Requirements_and_methods_for_Interception_Caching

NOTE: NAT configuration will only work when used on the squid box. This is
required to perform intercept accurately and securely. To intercept from a
gateway machine and direct traffic at a separate squid box use policy
routing.

I have been struggling for 2 days to setup a working Squid 4.8 or higher
interception.
Test server is running Ubuntu 18.4.3 and Squid 4.8.
Documentation is either too much trim or extremely outdated.
Any help would be very much appreciated.

policy routing:

https://wiki.squid-cache.org/ConfigExamples/Intercept/IptablesPolicyRoute

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
The only substitute for good manners is fast reflexes.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Problem caching range requests with range_offset_limit

[squid-users] Problem caching range requests with range_offset_limit

Re: [squid-users] SQL DB squid.conf backend, who was it that asked about it?

Re: [squid-users] Squid 4.8+ intercept

Re: [squid-users] Squid 4.8+ intercept

Re: [squid-users] Squid 4.8+ intercept

Re: [squid-users] Squid 4.8+ intercept

Re: [squid-users] Squid 4.8+ intercept

8 matches

Site Navigation

Mail list logo

Footer information