If you want to have a URL filter and to be able to reload
its configuration without a disruption of service,
one may use ufdbGuard. It is free Open Source Software and can be used with
your own URL database, free URL databases and commercial URL databases.
It is not very interesting which DB ufdbGuard uses. The unofficial name
is "ufdb 2.0". A utility called ufdbGenTable converts a text file to a
.ufdb URL table file which ufdbGuard reads.
For those who wish to use the core functionality of ufdbguard, there
is also an API available.
Marcus
On 06/10/2013 06:44 PM, Eliezer Croitoru wrote:
Please share these solutions with us..
I was working on a KV DB using Tokyo Cabinet, Tokyo tyrant, MongoDB, Redis and
more.
if you have something that do exists and can be used I will be happy to leave
this job to the pros.
Eliezer
On 6/11/2013 12:03 AM, Jose-Marcio Martins wrote:
Welllll... sorry for the top post...
If the "filter" is an external handler process... it should be able to
do all the job of updating its database, in memory or file based,
without boring squid, and without (or eventually a minimal) interruption
of service
There are some solutions out there...
Or maybe I didn't understood what you're talking about.
On 06/10/2013 05:43 PM, Squidblacklist wrote:
On Mon, 10 Jun 2013 12:16:40 -0300
Marcus Kool <marcus.k...@urlfilterdb.com> wrote:
[discussion about proposal 1 deleted]
About solution 2:
Consider the following scenario:
Suppose the parent proxy configuration must be reloaded.
What mechanism will be used to signal the child proxy to ignore
the parent?
Squid does this on its own. thats what I have been trying to tell
you. the child proxy knows to bypass the parent when it is
unavailable.(IE During reload or restart)
The child knows how to deal with a non-responsive parent. correct.
But in the process of recovering from a parent that suddenly does not
respond any more, CONNECT tunnel break, and HTTP object retrieval and
uploads in progress break. The client has no way of redoing or
repairing this.
- reload its configuration? No, reconfiguration of the client
stops all traffic.
Not if your directing your traffic to a child proxy, and reloadong
on the parent proxy.
The question was: how is the child signalled that the parent is
reconfiguring, with the intent to stop using the parent neatly and to
prevent that HTTP traffic in progress is processed without
interruption of service. The option to reload the configuration of
the client proxy does not work, since reconfiguration of a squid
proxy causes interruption of service. Especially when all traffic is
redirected to the client proxy.
- simply let the connection to the parent fail? this will lead to
timeouts and everything in progress fails.
Nothing fails in this configuration.
Have you tested this? In a live situation where applications use
CONNECT tunnels, HTTP POST with a large body, chat applications which
use a protocol where an HTTP GET may get a very late answer? And
what about applications that rely on persistent HTTP connections?
Yes this would seem to be a problem. I just confirmed.
- use more than 1 parent? can be done but is no cost effective
since one needs an extra Squid server and still everything in
progress fails. If I am missing something, please explain how the
child ignores the parent without interruption of service.
There is no added cost, you can run multiple instances of squid on
the same machine, by using a different conf and cache dirs for each
instance.
Squid is used in many institutions with a large configuration: large
memory and large caches. It is not obvious that institutions which
sized their environment for a particular task can run two Squid
proxies (parent and child) on the same hardware.
Well , I would argue that if you in fact setup a child, parent proxy,
resource requirements would be minimal for the child as it likely
wouldnt require any filtering or much resources.
Marcus
PS: what is name? Is it Ben or Fix ?
My name is Ben. If calling me Fix seems
silly, just use Ben.
My purpose in interjecting into your thread was not to disrupt or
dissuade discussion about improvements to squid proxy, I merely was
explaining the work arounds I see. And yes, it is not
perfect. At every price point there is an appropriate solution. And
while I admit, this economical solution may work for those who have no
alternative might not be acceptable for some,
In conclusion, I too would like to see a
true fix for squid that allows a reload without interrupting traffic,
or any sort of "work around".
Also, URLfilterDB looks like an excellent product.
-
Signed,
Fix Nichols
http://www.squidblacklist.org
-
Signed,
Fix Nichols
http://www.squidblacklist.org
