[squid-users] Anyone know where surftrackr has gone?
Hi, I have been looking into various analysis and reporting tools for squid logs. I have tried both webalizer and sarg - and both have strengths - and (for me) weaknesses. I have seen references to squidalyser which I think became Surftrackr - and I have found references to new releases as recently as march 2008 - but all the download links I try seem to be dead. Ideally I would like to find an RPM package - but source would also be fine. Does anyone know anything about this package - and where I can get it? Thanks Richard.
[squid-users] Help with sarg usage
Hi I have satrg installed and working - but have not found much documentation other than the man pages - which are fairly brief. Can anyone help me with these issues with sarg. 1) It appears to only use the current squid log by default - and the documentation doesn't seem to tell me how I can get it to read several squid log files. 2) When I first installed it - and told it to place reports in /var/ww/html/sarg - I could browse the report fine as http://192.168.0.201/sarg - but for some reason - I now get a Forbidden error. 3) How do you access the reports at the default location: /var/www/sarg? 4) If I specify the -t HH-HH option to restrict the report to a time range - it doesn't seem to behave as I would expect. I get far less traffic reported than I would expect over the period. I can't find any way to check that it is reporting all the relevant trafic. Thanks Richard.
Re: [squid-users] How to keep session?
On lör, 2008-06-14 at 23:33 +0800, Ken W. wrote: Can anybody tell me why source-hash algorithm in cache_peer directive can't handle the session case? Who said it can't? Since source-hash (if I understand for it correctly) will redirect users' requests based on their IPs, so the same IP will always go to the same original server. That will keep session always valid. Yes, unless the user is behind a cache mesh (i.e. a farm of Squid proxies). Regards Henrik signature.asc Description: This is a digitally signed message part
Re: [squid-users] what can I help to make this swap.state corruption go away?
On lör, 2008-06-14 at 13:38 -0300, Michel (M) wrote: friends this swap.state corruption is getting worse and worse and it is a pattern swap.state.new EVER stops at 72 and then one swap.state after the other grows until the disk is full This problem has only shown up in your installation. In all my tests and for all the years I have been supporting Squid users you are the only one who have encountered this. It might be something simple that differs between your setup and everyone else. Question is what.. Regards Henri signature.asc Description: This is a digitally signed message part
Re: [squid-users] Anyone know where surftrackr has gone?
On sön, 2008-06-15 at 14:57 +0800, Richard Chapman wrote: I have been looking into various analysis and reporting tools for squid logs. I have tried both webalizer and sarg - and both have strengths - and (for me) weaknesses. I have seen references to squidalyser which I think became Surftrackr - and I have found references to new releases as recently as march 2008 - but all the download links I try seem to be dead. Have you tried contacting the author? Regards Henrik signature.asc Description: This is a digitally signed message part
Re: [squid-users] cached MS updates !
no one explane this :(((
pokeman wrote:
hi there
Refrence to following atricle
http://www.nabble.com/Re%3A-YouTube-and-other-streaming-media-%28caching%29-p17738020.html
i am going to cached windowsupdate object here is changes in store script
in squid.conf and output log
Squid.conf
acl store_rewrite_list url_regex ^http://(.*?)/windowsupdate\?
refresh_pattern windowsupdate.com/.*\.(cab|exe|dll) 10080 90% 99
ignore-no-cache override-expire ignore-private
refresh_pattern download.microsoft.com/.*\.(cab|exe|dll) 10080 90% 99
ignore-no-cache override-expire ignore-private
refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe|psf) 10080 90%
99 ignore-no-cache override-expire ignore-private
refresh_pattern ^http://sjl-v[0-9]+\.sjl\.youtube\.com 10080 90% 99
ignore-no-cache override-expire ignore-private
#Store script
#!/usr/bin/perl
$|=1;
while () {
@X = split;
$url = $X[0];
$url =~
[EMAIL PROTECTED]://(.*?)/get_video\?(.*)video_id=(.*?)[EMAIL
PROTECTED]://videos.youtube.INTERNAL/ID=$3@;
$url =~
[EMAIL PROTECTED]://(.*?)/get_video\?(.*)video_id=(.*?)[EMAIL
PROTECTED]://videos.youtube.INTERNAL/ID=$3@;
$url =~
[EMAIL PROTECTED]://(.*?)/videodownload\?(.*)docid=(.*?)[EMAIL
PROTECTED]://videos.google.INTERNAL/ID=$3@;
$url =~
[EMAIL PROTECTED]://(.*?)/videodownload\?(.*)docid=(.*?)[EMAIL
PROTECTED]://videos.google.INTERNAL/ID=$3@;
$url =~
[EMAIL PROTECTED]://(.*?)/update\?(.*)video_id=(.*?)[EMAIL
PROTECTED]://au.download.windowsupdate.com.INTERNAL/ID=$3@;
$url =~
[EMAIL PROTECTED]://(.*?)/update\?(.*)video_id=(.*?)[EMAIL
PROTECTED]://au.download.windowsupdate.com.INTERNAL/ID=$3@;
print $url\n;
}
### output cache log
1213248096.431606 192.168.0.5 TCP_MISS/206 15348 GET
http://au.download.windowsupdate.com/msdownload/update/v5/psf/windowsxp-kb902400-x86-enu_a7c593892442e90b74d93abf0524a52f00998cea.psf
- DIRECT/199.93.42.124 application/octet-stream
1213248098.070905 192.168.0.5 TCP_MISS/206 36487 GET
http://au.download.windowsupdate.com/msdownload/update/v5/psf/windowsxp-kb902400-x86-enu_a7c593892442e90b74d93abf0524a52f00998cea.psf
- DIRECT/199.93.42.124 multipart/byteranges
1213248099.996 1535 192.168.0.5 TCP_MISS/206 40838 GET
http://au.download.windowsupdate.com/msdownload/update/v5/psf/windowsxp-kb902400-x86-enu_a7c593892442e90b74d93abf0524a52f00998cea.psf
- DIRECT/8.12.137.30 multipart/byteranges
1213248101.372 1216 192.168.0.5 TCP_MISS/206 14687 GET
http://au.download.windowsupdate.com/msdownload/update/v5/psf/windowsxp-kb902400-x86-enu_a7c593892442e90b74d93abf0524a52f00998cea.psf
- DIRECT/209.84.7.123 application/octet-stream
1213248101.749202 192.168.0.5 TCP_MISS/200 375 HEAD
http://download.windowsupdate.com/v7/windowsupdate/redir/wuredir.cab?0806120622
- DIRECT/79.140.80.33 application/octet-stream
1213248102.091606 192.168.0.5 TCP_MISS/206 12929 GET
http://au.download.windowsupdate.com/msdownload/update/v5/psf/windowsxp-kb902400-x86-enu_a7c593892442e90b74d93abf0524a52f00998cea.psf
- DIRECT/209.84.7.123 application/octet-stream
1213248103.755962 192.168.0.5 TCP_MISS/206 33762 GET
http://au.download.windowsupdate.com/msdownload/update/v5/psf/windowsxp-kb902400-x86-enu_a7c593892442e90b74d93abf0524a52f00998cea.psf
- DIRECT/209.84.7.123 multipart/byteranges
1213248104.624578 192.168.0.5 TCP_MISS/200 375 HEAD
http://www.update.microsoft.com/v7/windowsupdate/selfupdate/wuident.cab?0806120623
- DIRECT/65.55.13.158 application/octet-stream
1213248104.831100 192.168.0.5 TCP_MISS/200 376 HEAD
http://download.windowsupdate.com/v7/windowsupdate/a/selfupdate/WSUS3/x86/Other/wsus3setup.cab?0806120623
- DIRECT/79.140.80.33 application/octet-stream
1213248105.266431 192.168.0.5 TCP_MISS/200 25737 GET
http://download.windowsupdate.com/v7/windowsupdate/a/selfupdate/WSUS3/x86/Other/wsus3setup.cab?0806120623
- DIRECT/79.140.80.33 application/octet-stream
1213248105.638 1529 192.168.0.5 TCP_MISS/206 36542 GET
http://au.download.windowsupdate.com/msdownload/update/v5/psf/windowsxp-kb902400-x86-enu_a7c593892442e90b74d93abf0524a52f00998cea.psf
- DIRECT/199.93.42.124 application/octet-stream
1213248106.175409 192.168.0.5 TCP_MISS/206 13595 GET
http://au.download.windowsupdate.com/msdownload/update/v5/psf/windowsxp-kb902400-x86-enu_a7c593892442e90b74d93abf0524a52f00998cea.psf
- DIRECT/199.93.42.124 multipart/byteranges
1213248106.595102 192.168.0.5 TCP_MISS/200 375 HEAD
http://download.windowsupdate.com/v7/windowsupdate/redir/wuredir.cab?0806120623
- DIRECT/79.140.80.33 application/octet-stream
1213248108.882 1832 192.168.0.5 TCP_MISS/206 45373 GET
http://au.download.windowsupdate.com/msdownload/update/v5/psf/windowsxp-kb902400-x86-enu_a7c593892442e90b74d93abf0524a52f00998cea.psf
- DIRECT/8.12.137.30 application/octet-stream
1213248109.603608 192.168.0.5 TCP_MISS/206 16132 GET
Re: [squid-users] cached MS updates !
On sön, 2008-06-15 at 06:40 -0700, pokeman wrote: no one explane this :((( Didn't understand there was a question. What is the question? 1213248111.272979 192.168.0.5 TCP_MISS/206 44097 GET http://au.download.windowsupdate.com/msdownload/update/v5/psf/windowsxp-kb902400-x86-enu_a7c593892442e90b74d93abf0524a52f00998cea.psf - DIRECT/8.12.137.30 multipart/byteranges I guess you ask whi this isn't cached, it's because it's a partial request, only requesting parts of the object, and Squid can not yet cache partial objects. Regards Henrik signature.asc Description: This is a digitally signed message part
RE: [squid-users] Is it possible to have squid as do Proxy and OWA/RPCoHTTPS accelerator?
I am trying to do the same thing. OWA works, but so far no joy with RPCoHTTP. Do I have to do something in OL to make it accept the certificate? The cert's are purchased from godaddy.com. For each, I appended the bundled gd_intermediate to the domain cert. Also, in the example config for OWA, I am confused by the following: acl OWA dstdomain owa_hostname cache_peer_access owa_hostname allow OWA Doesn't the 2nd line just grant access from owa_hostname to owa_hostname ?? My current config (which works for OWA, but not RPCoHTTP): extension_methods RPC_IN_DATA RPC_OUT_DATA https_port public_ip_for_owa:443 cert=/usr/share/ssl/owa/combined.crt key=/usr/share/ssl/owa/owa.key defaultsite=owa.tld.com https_port public_ip_for_rpc:443 cert=/usr/share/ssl/rpc/combined.crt key=/usr/share/ssl/rpc/rpc.key defaultsite=rpc.tld.com cache_peer ip_of_exchange parent 80 0 no-query originserver front-end-https=auto login=PASS acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl to_localhost dst 127.0.0.0/8 acl SSL_ports port 443 563 acl CONNECT method CONNECT acl OWA dstdomain owa.tld.com acl RPC dstdomain rpc.tld.com http_access allow manager localhost http_access allow OWA http_access allow RPC http_access deny manager http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access deny to_localhost http_access allow localhost http_access deny all http_reply_access allow all icp_access deny all miss_access allow OWA miss_access allow RPC miss_access deny all cache_peer_access ip_of_exhcange allow OWA cache_peer_access ip_of_exhcange allow RPC cache_peer_access ip_of_exhcange deny all never_direct allow OWA never_direct allow RPC Thanks again, Alan Lehman -Original Message- From: Odhiambo Washington [mailto:[EMAIL PROTECTED] Sent: Monday, June 02, 2008 11:41 AM To: Squid users Subject: Re: [squid-users] Is it possible to have squid as do Proxy and OWA/RPCoHTTPS accelerator? On Mon, Jun 2, 2008 at 7:27 PM, Henrik Nordstrom [EMAIL PROTECTED] wrote: On mån, 2008-06-02 at 13:41 +0300, Odhiambo Washington wrote: (actually, this is supposed to be the only entry for cache_peer I am goingto have?) If you only have one server, and that server is only talking http then yes there is only a single cache_peer.. Understood. That has worked. It also requied a PEM passphrase. I hope this is not supposed to be another problem. These ssl stuff! You can configure the password in squid.conf if the PEM key is encrypted, or easily decrypt it with the openssl rsa command. Understood as well. In my case, I don't have a certificate for the external hostname, which brings me back to the confusing issue regarding the certificate: I can make a self-signed certificate for the external hostname. Not a problem. However, does this mean I really don't need the internal certifcate Exchange is using? Correct. Pooh! That was so confusing:-) Suppose: My Squid host is publicly known as mail.odhiambo.COM (IP of 1.2.3.4) My Exchange server is named msexch.msexch.odhiambo.BIZ (IP of 192.168.0.26) Given that both OWA and RPCoHTTPS are directed at these... What values should I use for the following variables (from the wiki): (a) owa_hostname? In https_port defaultsite you should use mail.odhiambo.COM as this is what the clients are expected to connect to. (b) ip_of_owa_server? The ip of your exchange/owa server. (c) rpcohttp.url.com? Ignore. That example uses a setup with more Exchange servers, where OWA is running on a separarate server from Exchange. (d) the_exchange_server? Ignore as above. From there, I believe I will only get stuck at the ssl certificates step, which is where I am still a bit confused. Since you are not going to use a real certificate then issue yourself a self-signed one using OpenSSL. openssl req -new -x509 -days 1 -nodes -out mail.odhiambo.COM_selfsigned.pem -keyout mail.odhiambo.COM_key.pem Everything is all clear now. Will find good time to test this out and see how well it goes. Thank you very much, Amos and Henrik! That was quite some hand-holding. I really appreciate. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223
Re: [squid-users] Anyone know where surftrackr has gone?
Hi Henrik I tried contacting him via an email address from the freshmeat website - and the email bounced. Any ideas where to find other contact details? Regards Richard. Henrik Nordstrom wrote: On sön, 2008-06-15 at 14:57 +0800, Richard Chapman wrote: I have been looking into various analysis and reporting tools for squid logs. I have tried both webalizer and sarg - and both have strengths - and (for me) weaknesses. I have seen references to squidalyser which I think became Surftrackr - and I have found references to new releases as recently as march 2008 - but all the download links I try seem to be dead. Have you tried contacting the author? Regards Henrik
[squid-users] Re: Help with sarg usage
Hi Richard, I hope this may help you. http://www.squid-cache.org/mail-archive/squid-users/200805/0172.html On Sun, Jun 15, 2008 at 12:33 PM, Richard Chapman [EMAIL PROTECTED] wrote: Hi I have satrg installed and working - but have not found much documentation other than the man pages - which are fairly brief. Can anyone help me with these issues with sarg. 1) It appears to only use the current squid log by default - and the documentation doesn't seem to tell me how I can get it to read several squid log files. 2) When I first installed it - and told it to place reports in /var/ww/html/sarg - I could browse the report fine as http://192.168.0.201/sarg - but for some reason - I now get a Forbidden error. 3) How do you access the reports at the default location: /var/www/sarg? 4) If I specify the -t HH-HH option to restrict the report to a time range - it doesn't seem to behave as I would expect. I get far less traffic reported than I would expect over the period. I can't find any way to check that it is reporting all the relevant trafic. Thanks Richard. -- Thank you Indunil Jayasooriya
Re: [squid-users] Is it possible to have squid as do Proxy and OWA/RPCoHTTPS accelerator?
Alan Lehman wrote: I am trying to do the same thing. OWA works, but so far no joy with RPCoHTTP. Do I have to do something in OL to make it accept the certificate? The cert's are purchased from godaddy.com. For each, I appended the bundled gd_intermediate to the domain cert. Also, in the example config for OWA, I am confused by the following: acl OWA dstdomain owa_hostname cache_peer_access owa_hostname allow OWA Doesn't the 2nd line just grant access from owa_hostname to owa_hostname ?? The two are independent things. The ACL dstdomain 'owa_hostname' is meant to be replaced by the FQDN of your public OWA which clients use to get to the service. The cache_peer_access owa_hostname is meant to be a seperate unique string 'X' exactly matching the value of the cache_peer name=X option. I've tweaked the wiki demo config a little to make that clear. My current config (which works for OWA, but not RPCoHTTP): extension_methods RPC_IN_DATA RPC_OUT_DATA https_port public_ip_for_owa:443 cert=/usr/share/ssl/owa/combined.crt key=/usr/share/ssl/owa/owa.key defaultsite=owa.tld.com https_port public_ip_for_rpc:443 cert=/usr/share/ssl/rpc/combined.crt key=/usr/share/ssl/rpc/rpc.key defaultsite=rpc.tld.com cache_peer ip_of_exchange parent 80 0 no-query originserver front-end-https=auto login=PASS You need a second entry for port 443 on the exchange server to handle the RPC requests. This is where the name= parameter becomes very important and needs to be unique for each entry and used in the cache_peer_access lines below. acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl to_localhost dst 127.0.0.0/8 acl SSL_ports port 443 563 acl CONNECT method CONNECT acl OWA dstdomain owa.tld.com acl RPC dstdomain rpc.tld.com http_access allow manager localhost http_access allow OWA http_access allow RPC http_access deny manager http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access deny to_localhost http_access allow localhost http_access deny all http_reply_access allow all icp_access deny all miss_access allow OWA miss_access allow RPC miss_access deny all cache_peer_access ip_of_exhcange allow OWA cache_peer_access ip_of_exhcange allow RPC cache_peer_access ip_of_exhcange deny all never_direct allow OWA never_direct allow RPC Thanks again, Alan Lehman -Original Message- From: Odhiambo Washington [mailto:[EMAIL PROTECTED] Sent: Monday, June 02, 2008 11:41 AM To: Squid users Subject: Re: [squid-users] Is it possible to have squid as do Proxy and OWA/RPCoHTTPS accelerator? On Mon, Jun 2, 2008 at 7:27 PM, Henrik Nordstrom [EMAIL PROTECTED] wrote: On mån, 2008-06-02 at 13:41 +0300, Odhiambo Washington wrote: (actually, this is supposed to be the only entry for cache_peer I am goingto have?) If you only have one server, and that server is only talking http then yes there is only a single cache_peer.. Understood. That has worked. It also requied a PEM passphrase. I hope this is not supposed to be another problem. These ssl stuff! You can configure the password in squid.conf if the PEM key is encrypted, or easily decrypt it with the openssl rsa command. Understood as well. In my case, I don't have a certificate for the external hostname, which brings me back to the confusing issue regarding the certificate: I can make a self-signed certificate for the external hostname. Not a problem. However, does this mean I really don't need the internal certifcate Exchange is using? Correct. Pooh! That was so confusing:-) Suppose: My Squid host is publicly known as mail.odhiambo.COM (IP of 1.2.3.4) My Exchange server is named msexch.msexch.odhiambo.BIZ (IP of 192.168.0.26) Given that both OWA and RPCoHTTPS are directed at these... What values should I use for the following variables (from the wiki): (a) owa_hostname? In https_port defaultsite you should use mail.odhiambo.COM as this is what the clients are expected to connect to. (b) ip_of_owa_server? The ip of your exchange/owa server. (c) rpcohttp.url.com? Ignore. That example uses a setup with more Exchange servers, where OWA is running on a separarate server from Exchange. (d) the_exchange_server? Ignore as above. From there, I believe I will only get stuck at the ssl certificates step, which is where I am still a bit confused. Since you are not going to use a real certificate then issue yourself a self-signed one using OpenSSL. openssl req -new -x509 -days 1 -nodes -out mail.odhiambo.COM_selfsigned.pem -keyout mail.odhiambo.COM_key.pem Everything is all clear now. Will find good time to test this out and see how well it goes. Thank you very much, Amos and Henrik! That was quite some hand-holding. I really appreciate. Amos -- Please use Squid 2.7.STABLE2 or 3.0.STABLE6
Re: [squid-users] Where are the ircache.net cgi for creating graphs?
What about this? http://www.squid-cache.org/~wessels/squid-rrd/ On Sat, Jun 14, 2008 at 3:23 AM, Richard Hubbell [EMAIL PROTECTED] wrote: Hello squid world, I was looking for the scripts that create the graphs on ircache.net, I found everything but the cgi scripts. Does anyone know where to get them? Or maybe there's another package that's preferred to make use of RRD for Squid? -- Thank you Indunil Jayasooriya
Re: [squid-users] Urgent Help Needed: Any suggestion on CPU utiliztion increased from 73% to 97.5 after cache_mem increased from 4 GB to 8 GB
Henrik, I used the memory cache only configuration without disk store So : cache_dir null /tmp On start up : Using 8192 Store buckets .. Is there any other information needed in order to analyze such problem ? Thanks , -Arkin On Sun, Jun 15, 2008 at 11:05 AM, Henrik Nordstrom [EMAIL PROTECTED] wrote: On sön, 2008-06-15 at 10:57 +0800, Arkin Y wrote: Acctually , I tried to use Linux OS default malloc, gnu malloc , dlmalloc, dlmalloc could help me get the best performance among the three different malloc . Interesting. For me dlmalloc fails completely when the process size goes above 2GB. I checked all my mgr:info log , when I set the cache_mem to different size , the stroe entries are different too . When the cache memory is 4GB , the Store Entries are ~519396 When the cache memory is 6 GB , the StoreEntries are ~776836 When the cache memory is 8 GB , the StoreEntries are ~1026097 Are you running without an on-disk store? (cache_dir) The average size of the stored objects almost ~8KB, so I am not sure whether such problem was caused by large objects . Ok. I checked a hash_lookup method , If in the worst cases in 8G scenario,it looks like almost 2 times of hash keys will be traversed to get the storeEntry than in 4 G scenario. The size of the hash table is dependent on the store size. But I guess this calculation screws up if there is no on-disk cache of reasonable size. On startup, what is reported NN for Using NN Store buckets in cache.log? Regards Henrik
[squid-users] iptables syntax
Hello, Now I have the plan to config a squid box for my office for web browsering. I will run Squid on Linux OS, with transparent mode. Should I use iptables to do the http intercept? what's the iptables syntax? please help, thank you. Regards, Ken W.
Re: [squid-users] iptables syntax
I will run Squid on Linux OS, with transparent mode. Should I use iptables to do the http intercept? what's the iptables syntax? please help, thank you. How many network card does this squid box have? in squid.conf, Pls add below line http_port 3128 transparent This is the iptables rule #Redirecting traffic destined to port 80 to port 3128 iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 80 -j REDIRECT --to-port 3128 for more, pls visit below URL http://wiki.squid-cache.org/SquidFaq/InterceptionProxy Happy Squiding -- Thank you Indunil Jayasooriya
