[squid-users] Looking for subscription plan plain text Blacklists for a spin

[Eliezer Croitoru]

Hey List,

 

I wanted to test my SquidBlocker DB newest version with a paid categories
list.

I am looking for a paid list of categorized sites which I can load into the
DB and test performance.

 

For now I have found the next list:

https://github.com/blocklistproject/Lists

 

But since there are many vendors for blacklists I would be happy to get any
recommendations.

 

Thanks,

Eliezer

 



Eliezer Croitoru

Tech Support

Mobile: +972-5-28704261

Email:   ngtech1...@gmail.com

Zoom: Coming soon

 

 

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] squid won't return cached even with refresh_pattern extra options override-lastmod override-expire ignore-reload ignore-no-store ignore-private store-stale

[Eliezer Croitoru]

Hey Mirek,

This is not the first time this issue rises.
There are risks in implementing any solution for this *issue*.

I have implemented YouTube caching in the past using couple twisted techniques 
while leaving squid un-touched.
The desire to caching sometimes can overcome couple very big risks to the 
integrity if the date/content.
It is possible to use an ICAP service with a 206 response instead of 204 or 200 
however I believe
that you wouldn’t need to cache any POST requests so a simple ICAP service 
would be sufficient.
I believe that It is preferred to leave squid sources un touched for such a 
purpose.
An example for such a twist is at:
* https://github.com/elico/squid-helpers/tree/master/squid_helpers/youtubetwist
* 
https://wiki.squid-cache.org/ConfigExamples/DynamicContent/Coordinator?highlight=%28cache_peer%29#Implementing_ICAP_solution
* https://ieeexplore.ieee.org/abstract/document/9072556

I wrote a public example of an ICAP server that was used to  prove 
vulnerabilities in HTTP which is now used in the prove of HTTPS vulnerabilities.
Take a peek at:
* https://github.com/elico/bgu-icap-example

It's written in GoLang and works under pretty heavy loads.

Let me know if you need more help,
Eliezer


Eliezer Croitoru
Tech Support
Mobile: +972-5-28704261
Email: ngtech1...@gmail.com
Zoom: Coming soon


-Original Message-
From: squid-users  On Behalf Of Alex 
Rousskov
Sent: Friday, March 26, 2021 10:36 PM
To: squid-users@lists.squid-cache.org
Subject: Re: [squid-users] squid won't return cached even with refresh_pattern 
extra options override-lastmod override-expire ignore-reload ignore-no-store 
ignore-private store-stale

On 3/24/21 3:34 PM, Miroslaw Malinowski wrote:
> I thought about upper service but as is not required at the moment,
> introducing extra hop just to remove the header looks a bit like a
> hammer approach. I'll look into how easily I can amend the code as the
> other option is to introduce a proxy like a feature to the application,
> so either way, it is a code change. The only problem here is that it's
> an OPNSense squid service so I have to compile from source on BSD and
> then keep adding in manually each time they do the update.

At the risk of stating the obvious: If your feature is officially
accepted into Squid sources, then you would not have to keep adding it
manually (once the changes reach your Squid packaging source).

Alex.


> On Wed, Mar 24, 2021 at 7:11 PM Alex Rousskov wrote:
> 
> On 3/24/21 2:49 PM, Miroslaw Malinowski wrote:
> 
> > looking at the code and reading carefully your response, you're saying
> > there is no way you can do it with squid.
> 
> With Squid, your options include:
> 
> 1. Squid source code changes. Should not be too difficult and, IMO, a
> high-quality implementation would deserve official acceptance because it
> is a generally useful feature in line with existing control knobs.
> 
> https://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F
> 
> 2. An adaptation service that removes Cache-Control:no-cache from the
> response before Squid processes it:
> https://wiki.squid-cache.org/SquidFaq/ContentAdaptation
> 
> 
> HTH,
> 
> Alex.
> 
> > On Wed, Mar 24, 2021 at 6:28 PM Miroslaw Malinowski wrote:
> >
> > Hi,
> >
> > You've right yes it's revalidating as API server I'm
> requesting data
> > is setting Cache-Control: no-cache. My question is how I can force
> > squid to cache and not validate as I know it's safe to do so. As
> > I've explained earlier we are making the same request and
> receiving
> > the same response from 100+ server so as to reduce number of
> > requests to the external server we would like squid to cache the
> > response and issue a cached version.
> >
> > 2021/03/24 18:00:54.867 kid1| 22,3| refresh.cc(351) refreshCheck:
> > YES: Must revalidate stale object (origin set no-cache or private)
> >
> > Mirek
> >
> > On Wed, Mar 24, 2021 at 6:15 PM Alex Rousskov
> >  
> >  >> wrote:
> >
> > On 3/24/21 12:48 PM, Miroslaw Malinowski wrote:
> >
> > > Probably, me missing on something silly or it can't be done
> > but I don't
> > > know why but squid won't return the cached version even
> when I
> > turn all
> > > override options ON in refresh_pattern.
> >
> > AFAICT, no configuration options that can disable
> revalidation of
> > Cache-Control:no-cache responses. refresh_pattern does not
> have an
> > (equivalent of) "ignore-no-cache-in-responses" option.
> >
> > IIRC, older Squids were v