Re: squid stabel 7

2009-10-20 Thread Chris Cappuccio 
You're using diskd but you didn't increase the sysv shared memory sizes.  The 
squid processes used sysv shared memory to talk to diskd.

There used to be some file with the squid port that told you what sizes to use 
for the sysv sysctls.  Maybe the sysv shared memory defaults were increased to 
the point where this became unnecessary ??

I've that squid likes to randomly crash with the memory protection in OpenBSD.  
I started noticing this after the new malloc was introduced.  Naturally, when 
the process crashes, it doesn't clean up its shared memory resource usage, so 
it restarts itself and uses more shared memory, eventually running out.  You 
have to use ipcs/ipcrm by hand to fix this problem each time it happens.

That pretty much makes 'diskd' mode unusable.  That's unfortunate because it is 
a very fast mode when you serve multiple concurrent cache users.  But it also 
points to bugs in squid that are unresolved, which sucks.  If you only care 
about proxying, you could try tinyproxy (which is what I use now)

sonjaya [sonj...@gmail.com] wrote:
 dear all
 
 i have openbsd in ibm 235 x series and  now running well with squid  ,
 i have problem running so sloow and have error message.
 
 2009/10/16 16:16:09| storeDiskdSend: msgsnd: (35) Resource temporarily
 unavailable
 2009/10/16 16:16:09| storeDiskdSend READ: (35) Resource temporarily 
 unavailable
 2009/10/16 16:16:09| storeUpdateCopy: Error at 155903 (-1)
 2009/10/16 16:16:09| storeDiskdSend: msgsnd: (35) Resource temporarily
 unavailable
 2009/10/16 16:16:09| storeDiskdSend CLOSE: (35) Resource temporarily 
 unavailable
 2009/10/16 16:16:09| storeSwapOutFileClosed: dirno 2, swapfile
 0D98, errflag=-1
 (35) Resource temporarily unavailable
 2009/10/16 16:16:09| storeDiskdSend: msgsnd: (35) Resource temporarily
 unavailable
 2009/10/16 16:16:09| storeDiskdSend UNLINK: (35) Resource temporarily
 unavailable
 2009/10/16 16:17:04| httpReadReply: Excess data from GET
 http://webcs.msg.yahoo.com/crossdomain.xml;
 
 - how to resolve this problem  ?
 - what the best cache system in openbsd ( diskd , ufs , auf 0 ?
 - Also methode like lru , heap , GDSF  in openbsd ?
 - how to increase max_filedescription in kernel openbsd , i only
 access by squid.conf ( check my squid.conf)
 - also anyone here a have link a step how to increase perfomance squid
 in  open bsd for a lot user ?
   i only get kernel tunning in freebsd , i check in my GENERIC not
 provide that informartion.
 
 before and after many thank's
 
 here may details machine :
 
 # squid -v
 Squid Cache: Version 2.7.STABLE6
 configure options:  '--datadir=/usr/local/share/squid'
 '--enable-auth=basic digest' '--enable-arp-acl'
 '--enable-basic-auth-helpers=NCSA YP LDAP'
 '--enable-digest-auth-helpers=password ldap' '--enable-delay-pools'
 '--enable-external-acl-helpers=ip_user unix_group ldap_group'
 '--enable-forw-via-db'
 '--enable-negotiate-auth-helpers=squid_kerb_auth'
 '--enable-pf-transparent' '--enable-removal-policies=lru heap'
 '--enable-ssl' '--enable-storeio=aufs ufs diskd null'
 '--with-pthreads' '--localstatedir=/var/squid'
 '--enable-follow-x-forwarded-for' '--enable-snmp'
 '--prefix=/usr/local' '--sysconfdir=/etc' '--mandir=/usr/local/man'
 '--infodir=/usr/local/info' 'CC=cc' 'CFLAGS=-O2 -pipe'
 'LDFLAGS=-L/usr/local/lib' 'CPPFLAGS=-I/usr/local/include'
 
 herer my squid.conf
 
 authenticate_cache_garbage_interval 3600 seconds
 authenticate_ttl 3600 seconds
 authenticate_ip_ttl 0 seconds
 authenticate_ip_shortcircuit_ttl 0 seconds
 acl parno url_regex /etc/squid/parno.txt
 acl blockfile urlpath_regex -i /etc/squid/blockfile.txt
 acl all src 0.0.0.0/0.0.0.0
 acl manager proto cache_object
 acl localhost src 127.0.0.1
 acl to_localhost dst 127.0.0.0/255.0.0.0
 acl localnet src 192.168.48.0/255.255.255.0
 acl SSL_ports port 443
 acl SSL_ports port 444
 acl Safe_ports port 80
 acl Safe_ports port 443
 acl Safe_ports port 1025-65535
 acl Safe_ports port 70
 acl Safe_ports port 210
 acl Safe_ports port 280
 acl Safe_ports port 488
 acl Safe_ports port 591
 acl Safe_ports port 777
 acl Safe_ports port 5050
 acl Safe_ports port 5060
 acl Safe_ports port 5000
 acl Safe_ports port 443
 acl Safe_ports port 70
 acl Safe_ports port 210
 acl Safe_ports port 1025-65535
 acl Safe_ports port 280
 acl Safe_ports port 488
 acl Safe_ports port 591
 acl Safe_ports port 777
 acl CONNECT method CONNECT
 acl shoutcast rep_header X-HTTP09-First-Line ^ICY.[0-9]
 acl apache rep_header Server ^Apache
 http_access Deny parno
 http_access Deny blockfile
 http_access Allow manager localhost
 http_access Deny manager
 http_access Deny !Safe_ports
 http_access Deny CONNECT !SSL_ports
 http_access Allow localnet
 http_access Deny all
 http_reply_access Allow all
 icp_access Allow localnet
 icp_access Deny all
 ident_lookup_access Deny all
 reply_body_max_size 0 Allow all
 follow_x_forwarded_for Deny all
 acl_uses_indirect_client on
 delay_pool_uses_indirect_client on
 log_uses_indirect_client on

Re: squid stabel 7

2009-10-20 Thread sonjaya 
at the end back to aufs and working fine today , but how to make my
squid become parent cache  so i can control policy for one machine to
another machine .
Because i have several squid with openbsd in head n branch.
thank's

squid stabel 7

2009-10-16 Thread sonjaya 
dear all

i have openbsd in ibm 235 x series and  now running well with squid  ,
i have problem running so sloow and have error message.

2009/10/16 16:16:09| storeDiskdSend: msgsnd: (35) Resource temporarily
unavailable
2009/10/16 16:16:09| storeDiskdSend READ: (35) Resource temporarily unavailable
2009/10/16 16:16:09| storeUpdateCopy: Error at 155903 (-1)
2009/10/16 16:16:09| storeDiskdSend: msgsnd: (35) Resource temporarily
unavailable
2009/10/16 16:16:09| storeDiskdSend CLOSE: (35) Resource temporarily unavailable
2009/10/16 16:16:09| storeSwapOutFileClosed: dirno 2, swapfile
0D98, errflag=-1
(35) Resource temporarily unavailable
2009/10/16 16:16:09| storeDiskdSend: msgsnd: (35) Resource temporarily
unavailable
2009/10/16 16:16:09| storeDiskdSend UNLINK: (35) Resource temporarily
unavailable
2009/10/16 16:17:04| httpReadReply: Excess data from GET
http://webcs.msg.yahoo.com/crossdomain.xml;

- how to resolve this problem  ?
- what the best cache system in openbsd ( diskd , ufs , auf 0 ?
- Also methode like lru , heap , GDSF  in openbsd ?
- how to increase max_filedescription in kernel openbsd , i only
access by squid.conf ( check my squid.conf)
- also anyone here a have link a step how to increase perfomance squid
in  open bsd for a lot user ?
  i only get kernel tunning in freebsd , i check in my GENERIC not
provide that informartion.

before and after many thank's

here may details machine :

# squid -v
Squid Cache: Version 2.7.STABLE6
configure options:  '--datadir=/usr/local/share/squid'
'--enable-auth=basic digest' '--enable-arp-acl'
'--enable-basic-auth-helpers=NCSA YP LDAP'
'--enable-digest-auth-helpers=password ldap' '--enable-delay-pools'
'--enable-external-acl-helpers=ip_user unix_group ldap_group'
'--enable-forw-via-db'
'--enable-negotiate-auth-helpers=squid_kerb_auth'
'--enable-pf-transparent' '--enable-removal-policies=lru heap'
'--enable-ssl' '--enable-storeio=aufs ufs diskd null'
'--with-pthreads' '--localstatedir=/var/squid'
'--enable-follow-x-forwarded-for' '--enable-snmp'
'--prefix=/usr/local' '--sysconfdir=/etc' '--mandir=/usr/local/man'
'--infodir=/usr/local/info' 'CC=cc' 'CFLAGS=-O2 -pipe'
'LDFLAGS=-L/usr/local/lib' 'CPPFLAGS=-I/usr/local/include'

herer my squid.conf

authenticate_cache_garbage_interval 3600 seconds
authenticate_ttl 3600 seconds
authenticate_ip_ttl 0 seconds
authenticate_ip_shortcircuit_ttl 0 seconds
acl parno url_regex /etc/squid/parno.txt
acl blockfile urlpath_regex -i /etc/squid/blockfile.txt
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1
acl to_localhost dst 127.0.0.0/255.0.0.0
acl localnet src 192.168.48.0/255.255.255.0
acl SSL_ports port 443
acl SSL_ports port 444
acl Safe_ports port 80
acl Safe_ports port 443
acl Safe_ports port 1025-65535
acl Safe_ports port 70
acl Safe_ports port 210
acl Safe_ports port 280
acl Safe_ports port 488
acl Safe_ports port 591
acl Safe_ports port 777
acl Safe_ports port 5050
acl Safe_ports port 5060
acl Safe_ports port 5000
acl Safe_ports port 443
acl Safe_ports port 70
acl Safe_ports port 210
acl Safe_ports port 1025-65535
acl Safe_ports port 280
acl Safe_ports port 488
acl Safe_ports port 591
acl Safe_ports port 777
acl CONNECT method CONNECT
acl shoutcast rep_header X-HTTP09-First-Line ^ICY.[0-9]
acl apache rep_header Server ^Apache
http_access Deny parno
http_access Deny blockfile
http_access Allow manager localhost
http_access Deny manager
http_access Deny !Safe_ports
http_access Deny CONNECT !SSL_ports
http_access Allow localnet
http_access Deny all
http_reply_access Allow all
icp_access Allow localnet
icp_access Deny all
ident_lookup_access Deny all
reply_body_max_size 0 Allow all
follow_x_forwarded_for Deny all
acl_uses_indirect_client on
delay_pool_uses_indirect_client on
log_uses_indirect_client on
ssl_unclean_shutdown off
sslproxy_version 1
http_port 0.0.0.0:3128 transparent protocol=http
zph_mode off
zph_local 0
zph_sibling 0
zph_parent 0
zph_option 136
dead_peer_timeout 10 seconds
hierarchy_stoplist cgi-bin
hierarchy_stoplist ?
cache_mem 8388608 bytes
maximum_object_size_in_memory 8192 bytes
memory_replacement_policy lru
cache_replacement_policy lru
cache_dir diskd /mnt/disk_cache/cache00 10024 16 256 Q1=72 Q2=64
cache_dir diskd /mnt/disk_cache/cache01 10024 16 256 Q1=72 Q2=64
cache_dir diskd /mnt/disk_cache/cache02 10024 16 256 Q1=72 Q2=64
store_dir_select_algorithm least-load
max_open_disk_fds 0
minimum_object_size 0 bytes
maximum_object_size 4194304 bytes
cache_swap_low 90
cache_swap_high 95
update_headers on
access_log /var/squid/logs/access.log squid
logfile_daemon /usr/local/libexec/logfile-daemon
cache_log /var/squid/logs/cache.log
cache_store_log /var/squid/logs/store.log
logfile_rotate 10
emulate_httpd_log off
log_ip_on_direct on
mime_table /etc/squid/mime.conf
log_mime_hdrs off
pid_filename /var/squid/logs/squid.pid
debug_options ALL,1
log_fqdn off
client_netmask 255.255.255.255
strip_query_terms on
buffered_logs off
netdb_filename

Re: squid stabel 7

2009-10-16 Thread Dimitri 
- how to resolve this problem  ?
I dont know

- what the best cache system in openbsd ( diskd , ufs , auf  0 ?
aufs is better because it manages to improve the performance multithreaded

- Also methode like lru , heap , GDSF  in openbsd ?

I use default lru

The cache replacement policy parameter determines which objects are evicted
(replaced) when disk space is needed.
   lru   : Squid's original list based LRU policy
   heap GDSF : Greedy-Dual Size Frequency
   heap LFUDA: Least Frequently Used with Dynamic Aging
   heap LRU  : LRU policy implemented using a heap

- how to increase max_filedescription in kernel openbsd , i only
access by squid.conf ( check my squid.conf)
I same

- also anyone here a have link a step how to increase perfomance squid in
openbsd for a lot user ?.

You need a personal tunning in squid.conf for example time of store image,
vmdeos etc.

For example:

cache_mem 64 MB
cache_swap_low 90
cache_swap_high 95
maximum_object_size 4096 KB
cache_dir aufs /var/spool/squid 1024 16 256
cache_access_log /var/log/squid/access.log
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log


See this  http://www.gulix.cl/wiki/Proxy_squid  Orly I have in spanish.
sorry.

Dimitri.-
http://deoxyt2.livejournal.com
OpenBSD - Free, Functional  Secure


--- El vie, 16/10/09, sonjaya sonj...@gmail.com escribis:

 De: sonjaya sonj...@gmail.com
 Asunto: squid stabel 7
 Para: OpenBSD-misc list misc@openbsd.org
 Fecha: viernes, 16 de octubre, 2009 06:43
 dear all

 i have openbsd in ibm 235 x series and  now running
 well with squid  ,
 i have problem running so sloow and have error message.

 2009/10/16 16:16:09| storeDiskdSend: msgsnd: (35) Resource
 temporarily
 unavailable
 2009/10/16 16:16:09| storeDiskdSend READ: (35) Resource
 temporarily unavailable
 2009/10/16 16:16:09| storeUpdateCopy: Error at 155903 (-1)
 2009/10/16 16:16:09| storeDiskdSend: msgsnd: (35) Resource
 temporarily
 unavailable
 2009/10/16 16:16:09| storeDiskdSend CLOSE: (35) Resource
 temporarily unavailable
 2009/10/16 16:16:09| storeSwapOutFileClosed: dirno 2,
 swapfile
 0D98, errflag=-1
 (35) Resource temporarily
 unavailable
 2009/10/16 16:16:09| storeDiskdSend: msgsnd: (35) Resource
 temporarily
 unavailable
 2009/10/16 16:16:09| storeDiskdSend UNLINK: (35) Resource
 temporarily
 unavailable
 2009/10/16 16:17:04| httpReadReply: Excess data from GET
 http://webcs.msg.yahoo.com/crossdomain.xml;

 - how to resolve this problem  ?
 - what the best cache system in openbsd ( diskd , ufs , auf
 0 ?
 - Also methode like lru , heap , GDSF  in openbsd ?
 - how to increase max_filedescription in kernel openbsd , i
 only
 access by squid.conf ( check my squid.conf)
 - also anyone here a have link a step how to increase
 perfomance squid
 in  open bsd for a lot user ?
   i only get kernel tunning in freebsd , i check in my
 GENERIC not
 provide that informartion.

 before and after many thank's

 here may details machine :

 # squid -v
 Squid Cache: Version 2.7.STABLE6
 configure options: 
 '--datadir=/usr/local/share/squid'
 '--enable-auth=basic digest' '--enable-arp-acl'
 '--enable-basic-auth-helpers=NCSA YP LDAP'
 '--enable-digest-auth-helpers=password ldap'
 '--enable-delay-pools'
 '--enable-external-acl-helpers=ip_user unix_group
 ldap_group'
 '--enable-forw-via-db'
 '--enable-negotiate-auth-helpers=squid_kerb_auth'
 '--enable-pf-transparent' '--enable-removal-policies=lru
 heap'
 '--enable-ssl' '--enable-storeio=aufs ufs diskd null'
 '--with-pthreads' '--localstatedir=/var/squid'
 '--enable-follow-x-forwarded-for' '--enable-snmp'
 '--prefix=/usr/local' '--sysconfdir=/etc'
 '--mandir=/usr/local/man'
 '--infodir=/usr/local/info' 'CC=cc' 'CFLAGS=-O2 -pipe'
 'LDFLAGS=-L/usr/local/lib' 'CPPFLAGS=-I/usr/local/include'

 herer my squid.conf

 authenticate_cache_garbage_interval 3600 seconds
 authenticate_ttl 3600 seconds
 authenticate_ip_ttl 0 seconds
 authenticate_ip_shortcircuit_ttl 0 seconds
 acl parno url_regex /etc/squid/parno.txt
 acl blockfile urlpath_regex -i /etc/squid/blockfile.txt
 acl all src 0.0.0.0/0.0.0.0
 acl manager proto cache_object
 acl localhost src 127.0.0.1
 acl to_localhost dst 127.0.0.0/255.0.0.0
 acl localnet src 192.168.48.0/255.255.255.0
 acl SSL_ports port 443
 acl SSL_ports port 444
 acl Safe_ports port 80
 acl Safe_ports port 443
 acl Safe_ports port 1025-65535
 acl Safe_ports port 70
 acl Safe_ports port 210
 acl Safe_ports port 280
 acl Safe_ports port 488
 acl Safe_ports port 591
 acl Safe_ports port 777
 acl Safe_ports port 5050
 acl Safe_ports port 5060
 acl Safe_ports port 5000
 acl Safe_ports port 443
 acl Safe_ports port 70
 acl Safe_ports port 210
 acl Safe_ports port 1025-65535
 acl Safe_ports port 280
 acl Safe_ports port 488
 acl Safe_ports port 591
 acl Safe_ports port 777
 acl CONNECT method CONNECT
 acl shoutcast rep_header X-HTTP09-First-Line ^ICY.[0-9]
 acl apache rep_header Server ^Apache
 http_access Deny parno
 http_access