The cpu is around 100% even no any requests is going.... For now I left just one cache_peer in configuration and got the new error:
commBind Cannot bind socket FD 28 to [::]: (13) Permission denied чт, 20 апр. 2023 г. в 17:21, Alex Rousskov <rouss...@measurement-factory.com >: > On 4/20/23 04:23, Alexeyяр Gruzdov wrote: > > > cache_peer peerG1.com parent 40001 0 no-query no-digest name=peerG1 > > > > external_acl_type ext_proxy_g1_type %LOGIN %DST /usr/local/bin/g1.py > > > > acl proxy_g1_ext_mark_acl ext_proxy_g1_type > > > > acl proxy_g1_ext_marked_acl annotate_transaction proxy=g1 > > > > acl proxy_peerG1_acl note proxy g1 > > > > http_access deny proxy_g1_ext_mark_acl proxy_g1_ext_marked_acl !all > > ..... > > others http_access rules > > > > And this above works. > > Glad to hear that. ( If others are going to use the above as a guiding > example, I would recommend naming these ACLs very differently, but that > is not important to Squid. ) > > > > BUT > > I am worried about why this my external script for ACL type loads the > > one of core of CPU to 100%.....??? > > External ACL caching aside, the script will be contacted once for every > Squid transaction. Does your script CPU usage go down to zero when there > is no traffic? If not, then there is a bug in the script itself. > > If you use the script from the command line, without Squid, does it > consume a lot of CPU and/or take a lot of time per fake query? You can > adjust the script to log the real query (when the script is used by > Squid), so that you can easily replicate that query when running the > script without Squid... > > The cache key in your case is (the expansion of) "%LOGIN %DST". It is > enabled by default IIRC. Look for "cache" related options at > http://www.squid-cache.org/Doc/config/external_acl_type/ > > > > ( I used three of workers in config, > > but I can see a six process called like my external helper script, looks > > like squid runs x2 process for external ACL ) > > See external_acl_type children-* options: > http://www.squid-cache.org/Doc/config/external_acl_type/ > > In most environments, I recommend setting all three of them to the same > value. Please note that these options are not SMP-aware (yet), so Squid > will _not_ divide their values by the number of workers and give each > worker as many children as you state in squid.conf. > > > > Because if I will put the one more group of users (that must to use > > another cache_peer ) - I will need to create one more external script > > that will making to check an existed users from an other DB table > > Once you get the basic setup above working for one group to your > satisfaction, I would recommend migrating from (one script and one > matching annotate_transaction ACL) per group to a single script for all > groups. That single external ACL script will send the right > annotation(s) to Squid. > > > HTH, > > Alex. > > > > ср, 19 апр. 2023 г. в 22:39, Alex Rousskov: > > > > On 4/19/23 13:30, Alexeyяр Gruzdov wrote: > > > > > cache_peer peerG1.com parent 40001 0 no-query no-digest > name=peerG1 > > > > > external_acl_type ext_proxy_g1_type %LOGIN %DST > /usr/local/bin/g1.py > > > > > acl proxy_g1_ext_acl ext_proxy_g1_type > > > > OK. I assume that /usr/local/bin/g1.py will only match users that > > should > > go to cache_peer called peerG1. > > > > > > > acl proxy_g1_ext_acl_mark annotate_transaction proxy=g1 > > > > Please note that the name of this annotate_transaction ACL -- > > "proxy_g1_ext_acl_mark" -- implies a relationship to the external ACL > > named "proxy_g1_ext_acl", but there is no such relationship. Squid > does > > not care about ACL names, but this naming problem may indicate a > > misunderstanding. To follow your naming scheme, this ACL should be > > called something like "proxy_g1_mark_acl" or "mark_for_proxy_g1_acl". > > > > > > > acl proxy_peerG1_acl note proxy g1 > > > > OK. FWIW, a more consistent ACL name would have been > > "proxy_g1_marked_acl" or "marked_for_proxy_g1_acl". Again, Squid does > > not really care about these names, so use whatever you think is > > consistent/meaningful/etc. > > > > > > > http_access deny proxy_g1_ext_acl !all > > > > This line has no (positive) effect. Squid will evaluate the external > > ACL, but since the rule, as a whole, will never match due to "!all", > > and > > since the external ACL has no (relevant) side effects, you can just > > delete this line from your configuration. > > > > Needless to say, if you delete this line, then proxy_g1_ext_acl will > be > > unused, which should tell you that this configuration is not doing > what > > you want. See below for a fix recommendation. > > > > > > > http_access deny proxy_g1_ext_acl_mark !all > > > > This line will mark _all_ transactions. You only want to mark > > transactions that also matched proxy_g1_ext_acl. That "b only if a" > > logic is accomplished by using _both_ ACLs in the same rule: > > > > http_access deny proxy_g1_ext_acl proxy_g1_ext_acl_mark !all > > > > With the above http_access rule (instead of the earlier two), Squid > > will > > evaluate the external ACL, and, if it matches, Squid will also > evaluate > > the annotation-setting ACL. The whole rule will then be rejected due > to > > "!all", but not until it annotates the transaction (if the external > ACL > > matches). Again, in this sketch, we are using this rule for its > > annotation side effect only. > > > > > > > And this works like I need now.... > > > > AFAICT, if the tests indicate that this configuration works, then the > > tests are broken. IMHO, you should fix the tests (while you have a > > broken configuration that can be used to test the tests) before > > proceeding with the configuration fix. > > > > > > HTH, > > > > Alex. > > P.S. Please keep this email thread on squid-users instead of > responding > > to me personally. > > > > > > > > > > > ср, 19 апр. 2023 г. в 21:01, Alexeyяр Gruzdov: > > > > > > so, ok - Lets limit just to one cache peer and one single > > ACL (just > > > to understand the logic): > > > > > > cache_peer peerG1.com parent 40001 0 no-query no-digest > > name=peerG1 > > > > > > external_acl_type ext_proxy_g1_type %LOGIN %DST > > > /usr/local/bin/g1.py (this will answer "OK" or "ERR", > > depends if > > > user consists in DB) > > > > > > acl proxy_g1_ext_acl ext_proxy_g1_type annotate_transaction > > > proxy=g1 (If I right understood here is a key point of how > > to add > > > the tag to transaction related with user) > > > acl proxy_peerG1_acl note proxy g1 (here we create the ACL > > based > > > on the tag and this is fast ACL yet and we should to use it in > > > cache_peer_access) > > > > > > > > > http_access deny proxy_g1_ext_acl !all > > > ......<others http access rules> > > > > > > > > > cache_peer_access peerG1 allow proxy_peerG1_acl > > > cache_peer_access peerG1 deny all > > > > > > Is that correct ? > > > > > > вт, 18 апр. 2023 г. в 23:44, Alex Rousskov > > > <rouss...@measurement-factory.com > > <mailto:rouss...@measurement-factory.com> > > > <mailto:rouss...@measurement-factory.com > > <mailto:rouss...@measurement-factory.com>>>: > > > > > > On 4/18/23 11:41, Alexeyяр Gruzdov wrote: > > > > > > > Could you explain me how the annotation transaction > > works and > > > how it > > > > related to acl that I could to use with cache_peers > > > > > > Transactions have a (possibly empty) set of name=value > > annotations. > > > > > > During Squid configuration time, Squid parses all ACL > > > declarations in > > > your configuration file. When Squid parses an > > > annotation_transaction ACL > > > declaration, Squid remembers what transaction annotation > > to add > > > in the > > > future, [every time] when that ACL is evaluated (e.g., > > used in > > > http_access rule that Squid reaches during transaction > > processing). > > > > > > When evaluated, an "annotation_transaction" ACL simply > > adds the > > > previously configured annotation to the current > > transaction and > > > returns > > > a "yes, this transaction matches" result. > > > > > > When evaluated, a "note" ACL returns a "yes, this > transaction > > > matches" > > > result if and only if the current transaction already has > the > > > matching > > > annotation. This ACL does not modify the set of > transaction > > > annotations. > > > > > > The combination of annotate_transaction and note ACLs > > allows you to > > > annotate a transaction at one time and check previously > set > > > transaction > > > annotations at another time. The timing and meaning of > those > > > annotations > > > are up to you. > > > > > > > > > > ok! Lets look to my case example: > > > > > > > cache_peer peerG1.com parent 40001 0 no-query no-digest > > > name=peerG1 round-robin > > > > > > > cache_peer_access peerG1 allow proxy_peerG1_acl > > > > cache_peer_access peerG1 allow proxy_all_acl > > > > cache_peer_access peerG1 deny all > > > > > > > acl proxy_peerG1_acl proxy_auth "../users.peerG1.txt" > > > > acl proxy_all_acl proxy_auth "../users.all.txt" > > > > > > [ I added the missing "acl " directive to the above ACL > > > declarations and > > > stripped rules for two out of three cache_peers ] > > > > > > As you know, the above cache_peer_access configuration is > not > > > supported > > > because it uses "slow" proxy_auth ACLs in > cache_peer_access > > > directives > > > that only support "fast" ACLs. It does not matter (to me), > > > whether the > > > above appears to "work" in some environments. YMMV. > > > > > > To fix this problem, we can use http_access rules to > > essentially > > > remember proxy_auth evaluation results (at http_access > > > evaluation time) > > > as transaction annotations. Here is an untested sketch > that > > > omits other > > > (important but irrelevant here) http_access rules and > assumes > > > that these > > > sketched http_access rules _are_ evaluated: > > > > > > # if proxy_peerG1_acl matches, evaluate > mark_for_peerG1 > > > http_access deny proxy_peerG1_acl mark_for_peerG1 !all > > > > > > # if proxy_all_acl matches, evaluate > mark_for_all_peers > > > http_access deny proxy_all_acl mark_for_all_peers !all > > > > > > > > > Now we can use those remembered proxy_... acl evaluation > > results > > > (i.e. > > > we can check for the matching annotations) in > > cache_peer_access > > > rules: > > > > > > cache_peer_access peerG1 allow marked_for_peerG1 > > > cache_peer_access peerG1 allow marked_for_all_peers > > > cache_peer_access peerG1 deny all > > > > > > > > > where the new ACLs mentioned above are declared along > > these lines: > > > > > > acl mark_for_peerG1 annotate_transaction for_peer_=G1 > > > acl mark_for_all_peers annotate_transaction > > for_all_peers_=true > > > > > > acl marked_for_peerG1 note for_peer_ G1 > > > acl marked_for_all_peers note for_all_peers_ true > > > > > > This can probably be simplified further by using > > for_peer_=ALL > > > instead > > > of for_all_peers_=true annotation, but I wanted to > > preserve the > > > symmetry > > > with your original configuration. > > > > > > > > > > And these all works like I need, But - once I am > > changing a > > > list of > > > > users (add or remove) - I need to use "squid -k > > > reconfigure"...... but > > > > of course better to go without this reconfigure > > > > > > One can avoid reconfiguration using an external ACL > > script that > > > gives > > > Squid the right for_peer_=... annotations (instead of > using > > > "constant" > > > or "hard-coded" annotate_transaction ACLs to store the > same > > > annotations). > > > > > > However, it may be better to make the above sketch to work > > > _before_ you > > > replace mark_for_peerG1 ACLs/rules with an external > > > mark_for_the_right_peer ACL. > > > > > > > > > HTH, > > > > > > Alex. > > > P.S. This thread continues the discussion started at > > > https://bugs.squid-cache.org/show_bug.cgi?id=5268 > > <https://bugs.squid-cache.org/show_bug.cgi?id=5268> > > > <https://bugs.squid-cache.org/show_bug.cgi?id=5268 > > <https://bugs.squid-cache.org/show_bug.cgi?id=5268>> > > > > > > _______________________________________________ > > > squid-users mailing list > > > squid-users@lists.squid-cache.org > > <mailto:squid-users@lists.squid-cache.org> > > > <mailto:squid-users@lists.squid-cache.org > > <mailto:squid-users@lists.squid-cache.org>> > > > http://lists.squid-cache.org/listinfo/squid-users > > <http://lists.squid-cache.org/listinfo/squid-users> > > > <http://lists.squid-cache.org/listinfo/squid-users > > <http://lists.squid-cache.org/listinfo/squid-users>> > > > > > > > > > > > > -- > > > С уважением к Вам > > > Алексей > > > +79043828661 > > > 620000 г.Екатеринбург 2022 > > > > > > > > > > > > -- > > > С уважением к Вам > > > Алексей > > > +79043828661 > > > 620000 г.Екатеринбург 2022 > > > > > > > > > > > -- > > С уважением к Вам > > Алексей > > +79043828661 > > 620000 г.Екатеринбург 2022 > > > > > > _______________________________________________ > > squid-users mailing list > > squid-users@lists.squid-cache.org > > http://lists.squid-cache.org/listinfo/squid-users > > _______________________________________________ > squid-users mailing list > squid-users@lists.squid-cache.org > http://lists.squid-cache.org/listinfo/squid-users >
_______________________________________________ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users