He Xavier, I believe that there are couple logformat options that you should add to understand better this issue. The first thing in this scenario is to have the squid.conf and access.log output.
If you have a load of about 100 rps you shouldn't be required to have more then 1-2 workers tops. To know the actual load you can dump the cache manager details. With the next script: https://gist.github.com/elico/8790bdc835d8e9ecbc57e72fc31effc0 it would be possible to probably dump all the relevant sections and much more. Just be advised to review them and the squid.conf and the access.log before send a link to a compressed archive on the public list. If you prefer you can send the link to there in a private email so I can try to review them. Depends on the proxy network access it would be pretty simple to understand where this 403 is coming from. It's also possible that it's from the origin server and not from the proxy. I assume this 403 is not based on the authentication level but it might be possible to use a special deny_info to "mark" the culprit deny acl in your squid.conf. Ie we can set a special response code in the ranges of 40x or 50x to make sure if it's from the proxy side. If I remember you are using basic authentication so it's pretty simple to write a threaded helper that will be responsible on all requests but I still need to see the squid.conf and access.log Eliezer ---- Eliezer Croitoru NgTech, Tech Support Mobile: +972-5-28704261 Email: ngtech1...@gmail.com Web: https://ngtech.co.il/ My-Tube: https://tube.ngtech.co.il/ -----Original Message----- From: squid-users <squid-users-boun...@lists.squid-cache.org> On Behalf Of Xavier Lecluse Sent: Friday, 9 September 2022 11:21 To: Alex Rousskov <rouss...@measurement-factory.com> Cc: squid-users@lists.squid-cache.org Subject: Re: [squid-users] [Troubleshoot] Squid 3.3 - Lots of 403 erros when reducing the workers number Hello Alex, Thanks for your answer. >> We are using two squid proxies (Squid 3.3) >Squid v3 is not officially supported. My answers below may apply to >Squid v3, but they are based on Squid v5+. >> In order to address some issues with Java clients, we tried to lower >> the worker directive from 8 to 1, because of the relative low number >> of simultaneous connections on our SSquid servers (about 100rq/s) >> After reducing the worker value to 1, and restarting the proxies, we >> observed a great number of 403 errors, so we decided to rollback to 8 >> workers. >> - How the number of workers and these 403 errors can be correlated ? >I do not know the exact correlation vector in your environment, but >fewer workers means, among other things, smaller _aggregate_ >authentication cache size and higher load on individual authentication >helpers. To pinpoint the correlation, we would need to know _why_ Squid >is generating 403 (Forbidden) errors. Is there a specific way to find the reason ? Do I need to activate a certain logging level to see these traces ? I will check in the actual logs and I'll come back if I don't find anything >> - Is there any "recommandations" about the number of workers to use, >> for a given number of request/s ? >Workers are primarily a performance optimization. For related tuning >suggestions, please see >https://wiki.squid-cache.org/Features/SmpScale#How_to_configure_SMP_Squid_for_top_performance.3F Yes, I saw and read this thread on the Wiki. We were already using some of these recommandations. I think we are quite far from a "top performance" but we didn't observed any performance issue when we used 8 workers. I understand that an individual worker should be able to handle our actual load, but we have to find what is provoking the 403 errors first. >> The inital problem is from some java clients, which are using two TCP >> sessions, one for the authentication, and another one for the HTTP(s) >> requests. The fact is that the "second" session is not always opened >> on the same worker, so ot considers that the authentication step has >> not already been done. >> Is there a way to address this issue ? >If (a request on) the second connection has enough information to link >it to the first/authenticated request/connection, then it may be >possible to configure Squid and write authentication helpers in such a >way that the "other" worker knows that the client of the second >connection has already authenticated. The details would depend on the >authentication scheme and that "linking" mechanism. Do you have any example of an authentication helper I can start with ? or an example on how to "link" events between workers ? I was thinking that workers were quite "individuals" and did not exchange anything (or verry little) with each other. Any start point would be welcome. Regards, Xavier >HTH, >Alex. ----- Mail original ----- De: "Alex Rousskov" <rouss...@measurement-factory.com> À: squid-users@lists.squid-cache.org Envoyé: Jeudi 8 Septembre 2022 17:19:19 Objet: Re: [squid-users] [Troubleshoot] Squid 3.3 - Lots of 403 erros when reducing the workers number On 9/8/22 10:15, Xavier Lecluse wrote: > We are using two squid proxies (Squid 3.3) Squid v3 is not officially supported. My answers below may apply to Squid v3, but they are based on Squid v5+. > In order to address some issues with Java clients, we tried to lower > the worker directive from 8 to 1, because of the relative low number > of simultaneous connections on our SSquid servers (about 100rq/s) > After reducing the worker value to 1, and restarting the proxies, we > observed a great number of 403 errors, so we decided to rollback to 8 > workers. > - How the number of workers and these 403 errors can be correlated ? I do not know the exact correlation vector in your environment, but fewer workers means, among other things, smaller _aggregate_ authentication cache size and higher load on individual authentication helpers. To pinpoint the correlation, we would need to know _why_ Squid is generating 403 (Forbidden) errors. > - Is there any "recommandations" about the number of workers to use, > for a given number of request/s ? Workers are primarily a performance optimization. For related tuning suggestions, please see https://wiki.squid-cache.org/Features/SmpScale#How_to_configure_SMP_Squid_for_top_performance.3F > The inital problem is from some java clients, which are using two TCP > sessions, one for the authentication, and another one for the HTTP(s) > requests. The fact is that the "second" session is not always opened > on the same worker, so ot considers that the authentication step has > not already been done. > Is there a way to address this issue ? If (a request on) the second connection has enough information to link it to the first/authenticated request/connection, then it may be possible to configure Squid and write authentication helpers in such a way that the "other" worker knows that the client of the second connection has already authenticated. The details would depend on the authentication scheme and that "linking" mechanism. HTH, Alex. _______________________________________________ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users _______________________________________________ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users _______________________________________________ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
