17.02.15 6:07, Marcus Kool пишет:
On 02/16/2015 08:00 PM, Eliezer Croitoru wrote:
Hey Yuri,
OK I have seen something...
Now we might need also the virtual memory which might be vsz.
And the cachemgr output is not from squidview..
The last image I have seen from cachemgr was much helpful(with 10
helpers).
From what I have seen until now squidGuard uses about 13 MB of ram
constantly.
If this is what it costs to run a squidGuard helper you should
consider that it has some peak usage time and lower bound usage time
and you will need to calculate how much helpers you need to provide
smooth surfing on peak usage time.
You should consider this peak usage as the required memory from
squidGuard for smooth operation.
Since not you and not will be able to change the basic memory
footprint(required) for squidGuard you will need to somehow decide if
you require a much efficient software or to adjust your system
accordingly.
I would agree that 100 * 13 MB of ram means about 1.3 GB of ram usage
and might not be acceptable for a "simple" filtering mechanism.
It is because squidGuard uses Berkeley DB which uses a default cache
size of 256 KB for each URL table.
And each squidGuard process does this. And of course the OS keeps a
full copy in the file system buffer cache.
So with 100 redirectors and N URL tables, N * 100 * 256K is used by
buffers for Berkeley DB alone.
The main reasons for the unknown need for 100 helpers might be since
it was not designed to be used this way.
squidGuard does not support the Squid feature 'concurrency' for
url_rewrite_children. ufdbGuard does.
With concurrency, latency goes down and the number of processes can
also be reduced.
The lack of concurrency is main disadvantage of squidGuard. Old
process-based model is not acceptable on modern OS'es.
BTW, can I configure ufdbGuard children like this:
url_rewrite_children 1 startup=1 idle=1 concurrency=100
?
Another reason is store ID rewriter too. With this we have the same
problem, but a bit smaller, of course.
I think, global parameter to firce shutdown idle redirectirs will be
best solution.
I must say that squidGuard is a very good piece of software but it
lacks couple things which might make it un-usable for some if not
many systems and maybe including yours.
Indeed in your case there is a chance that if you will even install a
full blown DB that will be stored all in ram and you will write a
helper that will mimic squidguard functionality\logic with
concurrency support you will get much less memory consumption with
much more efficient request handling.
It is much smarter to write a helper which will have a overall
efficiency mark on it then to add into squid something that might not
be even needed in the first place.
ufdbGuard has all that you need. It holds one copy of the database in
memory without allocating extra memory.
The ufdbGuard URL redirectors are leightweight processes using very
little memory and
the redirectors support concurrency so less are required.
Off list, Yuri asked help for compilation on Solaris and I made a fix
for ufdbGuard which is availabe for Yuri.
If there are no further complaints from Yuri, I will release the patch
in one or two days.
Will try. Also will be good to support modern startup facility on
Solaris ;) against deprecated SVR4 init. ;)
Marcus
For now before writing any helper etc I will want to see the cachemgr
interface output for the "redirector" option in two cases:
- startup
- after midnight when there are lots of helpers running under squid
You can see that squid since version 3.2 squid offers access to the
squid cache manager interface using a simple url such as:
"http://filter:3128/squid-internal-mgr/redirector";
(using the visible_hostname and the forward proxy port of squid)
All The Bests,
Eliezer
On 16/02/2015 21:56, Yuri Voinov wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Now:
2015/02/16 23:10:23 kid1| store_swap_size = 29826351.50 KB
2015/02/16 23:10:24 kid1| storeLateRelease: released 0 objects
2015/02/16 23:15:01 kid1| Starting new redirector helpers...
2015/02/16 23:15:01 kid1| helperOpenServers: Starting 1/100
'squidGuard' processes
2015/02/17 01:40:15 kid1| Starting new redirector helpers...
2015/02/17 01:40:15 kid1| helperOpenServers: Starting 1/100
'squidGuard' processes
^Croot @ cthulhu / # ps -e -o user,pid,rss,comm
_______________________________________________
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
