[squid-users] storeDiskdSend OPEN: (35) Resource temporarily unavailable
Hi all, I'm running squid 2.6-stable17 on Freebsd 6.3. Machine is a 500MHz with 512MB RAM. ./configure --prefix=/usr/local/squid --enable- storeio=ufs,coss,diskd,null --enable-underscores --with-large-files -- enable-large-cache-files --enable-delay-pools --disable-ident-lookups --enable-snmp --enable-cache-digests --enable-underscores --enable- kill-parent-hack --enable-removal-policies --enable-async-io -- enable-kqueue --enable-follow-x-forwarded-for I just have squid displaying the following error message: 2008/01/20 18:24:15| storeDiskdSend OPEN: (35) Resource temporarily unavailable 2008/01/20 18:24:15| storeDiskdSend: msgsnd: (35) Resource temporarily unavailable 2008/01/20 18:24:15| assertion failed: diskd/store_io_diskd.c:541: "+ +send_errors < 100" jubilee# ipcs -a Message Queues: T ID KEY MODEOWNERGROUPCREATOR CGROUP CBYTES QNUM QBYTESLSPIDLRPID STIMERTIMECTIME q 524288 942080 --rwa-- nobody nobody nobody nobody 1280 40 2048 920 927 18:05:08 18:05:08 7:34:35 q 524289 942081 --rwa-- nobody nobody nobody nobody00 2048 927 920 18:05:08 18:05:08 7:34:35 Shared Memory: T ID KEY MODEOWNERGROUPCREATOR CGROUP NATTCHSEGSZ CPID LPID ATIME DTIMECTIME m 524288 942082 --rw--- nobody nobody nobody nobody1 339968 920 927 7:34:35 18:05:28 7:34:35 Semaphores: T ID KEY MODEOWNERGROUPCREATOR CGROUP NSEMS OTIMECTIME There's only 4 users on this server, and it's been running for quite sometime now. I read that I need to increase the message queue limits, Im presuming it's the "kern.ipc.msgmnb=16384". How can I monitor system before anything happens. I can run squidclient mgr:info, but what do I need to look for. Configured thee system with following parameters: kern.ipc.nmbclusters: 65536 kern.maxfiles=65536 kern.maxfilesperproc=32768 net.inet.ip.portrange.last=65535 kern.ipc.somaxconn=2048 kern.maxvnodes=10 kern.ipc.msgmnb=16384 kern.ipc.msgmni=40 kern.ipc.msgseg=512 kern.ipc.msgssz=64 kern.ipc.msgtql=2048 options SHMSEG=16 options SHMMNI=41 options MSGSSZ=64 options MSGTQL=512 options MSGSEZ=2048 options SHMMNI=40 options SHMMAX=2097152 options SHMALL=4096 options MAXFILES=8192 options NMBCLUSTERS=32768 options MSGMNB=16384 options VFS_AIO In my squid.conf: cache_dir diskd /usr/local/squid/var/cache 28000 32 512 Q1=72 Q2=64 Thanks BSD Networking, Microsoft Notworking
Re: [squid-users] storeDiskdSend OPEN: (35) Resource temporarily unavailable
Hm! Well, I can't help you with the diskd issue, but diskd is still unstable under high load. You're probably better off using aufs on FreeBSD-6.3. Adrian On Sun, Jan 20, 2008, Monah Baki wrote: > Hi all, > > I'm running squid 2.6-stable17 on Freebsd 6.3. Machine is a 500MHz > with 512MB RAM. > > ./configure --prefix=/usr/local/squid --enable- > storeio=ufs,coss,diskd,null --enable-underscores --with-large-files -- > enable-large-cache-files --enable-delay-pools --disable-ident-lookups > --enable-snmp --enable-cache-digests --enable-underscores --enable- > kill-parent-hack --enable-removal-policies --enable-async-io -- > enable-kqueue --enable-follow-x-forwarded-for > > > I just have squid displaying the following error message: > > 2008/01/20 18:24:15| storeDiskdSend OPEN: (35) Resource temporarily > unavailable > 2008/01/20 18:24:15| storeDiskdSend: msgsnd: (35) Resource > temporarily unavailable > 2008/01/20 18:24:15| assertion failed: diskd/store_io_diskd.c:541: "+ > +send_errors < 100" > > > jubilee# ipcs -a > Message Queues: > T ID KEY MODEOWNERGROUPCREATOR > CGROUP CBYTES QNUM > QBYTESLSPIDLRPID STIMERTIMECTIME > q 524288 942080 --rwa-- nobody nobody nobody > nobody 1280 40 2048 > 920 927 18:05:08 18:05:08 7:34:35 > q 524289 942081 --rwa-- nobody nobody nobody > nobody00 2048 > 927 920 18:05:08 18:05:08 7:34:35 > > Shared Memory: > T ID KEY MODEOWNERGROUPCREATOR > CGROUP NATTCHSEGSZ CPID LPID ATIME > DTIMECTIME > m 524288 942082 --rw--- nobody nobody nobody > nobody1 339968 920 927 7:34:35 > 18:05:28 7:34:35 > > Semaphores: > T ID KEY MODEOWNERGROUPCREATOR > CGROUP NSEMS OTIMECTIME > > > > There's only 4 users on this server, and it's been running for quite > sometime now. > > I read that I need to increase the message queue limits, Im presuming > it's the "kern.ipc.msgmnb=16384". How can I monitor system before > anything happens. I can run squidclient mgr:info, but what do I need > to look for. > > Configured thee system with following parameters: > > kern.ipc.nmbclusters: 65536 > kern.maxfiles=65536 > kern.maxfilesperproc=32768 > net.inet.ip.portrange.last=65535 > kern.ipc.somaxconn=2048 > kern.maxvnodes=10 > kern.ipc.msgmnb=16384 > kern.ipc.msgmni=40 > kern.ipc.msgseg=512 > kern.ipc.msgssz=64 > kern.ipc.msgtql=2048 > > options SHMSEG=16 > options SHMMNI=41 > options MSGSSZ=64 > options MSGTQL=512 > options MSGSEZ=2048 > options SHMMNI=40 > options SHMMAX=2097152 > options SHMALL=4096 > options MAXFILES=8192 > options NMBCLUSTERS=32768 > options MSGMNB=16384 > options VFS_AIO > > In my squid.conf: > cache_dir diskd /usr/local/squid/var/cache 28000 32 512 Q1=72 Q2=64 > > > > Thanks > > > BSD Networking, Microsoft Notworking > > -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -
Re: [squid-users] storeDiskdSend OPEN: (35) Resource temporarily unavailable
On Mon, Jan 21, 2008, Tek Bahadur Limbu wrote: > Hi Monah, > > Monah Baki wrote: > >Hi all, > > > >I'm running squid 2.6-stable17 on Freebsd 6.3. Machine is a 500MHz with > >512MB RAM. > > Don't you think that your system is a little low on resources even for a > low number of users? :P I dunno, I'm getting my Squid development branch to handle ~30mbit on a piii-600 celeron class box. Its just a shame the disk code is horrible at the moment :) Adrian
Re: [squid-users] storeDiskdSend OPEN: (35) Resource temporarily unavailable
Hi Monah, Monah Baki wrote: Hi all, I'm running squid 2.6-stable17 on Freebsd 6.3. Machine is a 500MHz with 512MB RAM. Don't you think that your system is a little low on resources even for a low number of users? ./configure --prefix=/usr/local/squid --enable-storeio=ufs,coss,diskd,null --enable-underscores --with-large-files --enable-large-cache-files --enable-delay-pools --disable-ident-lookups --enable-snmp --enable-cache-digests --enable-underscores --enable-kill-parent-hack --enable-removal-policies --enable-async-io --enable-kqueue --enable-follow-x-forwarded-for I think it's better to use "--enable-storeio=ufs,aufs,coss,diskd,null" and remove "--enable-async-io". I just have squid displaying the following error message: 2008/01/20 18:24:15| storeDiskdSend OPEN: (35) Resource temporarily unavailable 2008/01/20 18:24:15| storeDiskdSend: msgsnd: (35) Resource temporarily unavailable 2008/01/20 18:24:15| assertion failed: diskd/store_io_diskd.c:541: "++send_errors < 100" I have seen these error logs due to DISKD in the past. It went away after I switched to UFS. jubilee# ipcs -a Message Queues: T ID KEY MODEOWNERGROUPCREATOR CGROUP CBYTES QNUM QBYTESLSPIDLRPID STIMERTIMECTIME q 524288 942080 --rwa-- nobody nobody nobody nobody 1280 40 2048 920 927 18:05:08 18:05:08 7:34:35 q 524289 942081 --rwa-- nobody nobody nobody nobody00 2048 927 920 18:05:08 18:05:08 7:34:35 Shared Memory: T ID KEY MODEOWNERGROUPCREATOR CGROUP NATTCHSEGSZ CPID LPID ATIME DTIMECTIME m 524288 942082 --rw--- nobody nobody nobody nobody1 339968 920 927 7:34:35 18:05:28 7:34:35 Semaphores: T ID KEY MODEOWNERGROUPCREATOR CGROUP NSEMS OTIMECTIME There's only 4 users on this server, and it's been running for quite sometime now. I read that I need to increase the message queue limits, Im presuming it's the "kern.ipc.msgmnb=16384". How can I monitor system before anything happens. I can run squidclient mgr:info, but what do I need to look for. Configured thee system with following parameters: kern.ipc.nmbclusters: 65536 kern.maxfiles=65536 kern.maxfilesperproc=32768 net.inet.ip.portrange.last=65535 kern.ipc.somaxconn=2048 kern.maxvnodes=10 kern.ipc.msgmnb=16384 kern.ipc.msgmni=40 kern.ipc.msgseg=512 kern.ipc.msgssz=64 kern.ipc.msgtql=2048 options SHMSEG=16 options SHMMNI=41 options MSGSSZ=64 options MSGTQL=512 options MSGSEZ=2048 options SHMMNI=40 options SHMMAX=2097152 options SHMALL=4096 options MAXFILES=8192 options NMBCLUSTERS=32768 options MSGMNB=16384 options VFS_AIO Did you compiled your kernel or are you loading these tunables dynamically? I really am not an expert on this kernel compilation with FreeBSD. But I use the following values: options SHMSEG=128 options SHMMNI=256 options SHMMAX=50331648 # max shared memory segment size (bytes) options SHMALL=16384 # max amount of shared memory (pages) options MSGMNB=16384 # max # of bytes in a queue options MSGMNI=48 # number of message queue identifiers options MSGSEG=768 # number of message segments options MSGSSZ=64 # size of a message segment options MSGTQL=4096 # max messages in system But I guess they are not relevant with 6.x because they are tunables which you can load dynamically. In the end if DISKD does not work for you, then I guess you should use UFS, AUFS or COSS. Since you only have 4 users, all of them will work fine for you... Thanking you... In my squid.conf: cache_dir diskd /usr/local/squid/var/cache 28000 32 512 Q1=72 Q2=64 Thanks BSD Networking, Microsoft Notworking -- With best regards and good wishes, Yours sincerely, Tek Bahadur Limbu System Administrator (TAG/TDG Group) Jwl Systems Department Worldlink Communications Pvt. Ltd. Jawalakhel, Nepal http://www.wlink.com.np http://teklimbu.wordpress.com
Re: [squid-users] storeDiskdSend OPEN: (35) Resource temporarily unavailable
Hi Tek, I am planning on moving 1500 users to the proxy, ofcourse the system will change (hardware wise). Performance of squid should be important, I knew diskd was still in "test" mode, but I did not expect 4 users even though it was running for 27 days to display this error messages. For my own info, if I move to ufs, and if ufs is pretty stable, performance wise is it close to diskd or better. Thanks On Jan 21, 2008, at 1:57 AM, Tek Bahadur Limbu wrote: Hi Monah, Monah Baki wrote: Hi all, I'm running squid 2.6-stable17 on Freebsd 6.3. Machine is a 500MHz with 512MB RAM. Don't you think that your system is a little low on resources even for a low number of users? ./configure --prefix=/usr/local/squid --enable- storeio=ufs,coss,diskd,null --enable-underscores --with-large- files --enable-large-cache-files --enable-delay-pools --disable- ident-lookups --enable-snmp --enable-cache-digests --enable- underscores --enable-kill-parent-hack --enable-removal-policies -- enable-async-io --enable-kqueue --enable-follow-x-forwarded-for I think it's better to use "--enable- storeio=ufs,aufs,coss,diskd,null" and remove "--enable-async-io". In the end if DISKD does not work for you, then I guess you should use UFS, AUFS or COSS. Since you only have 4 users, all of them will work fine for you... Thanking you... In my squid.conf: cache_dir diskd /usr/local/squid/var/cache 28000 32 512 Q1=72 Q2=64 Thanks BSD Networking, Microsoft Notworking -- With best regards and good wishes, Yours sincerely, Tek Bahadur Limbu System Administrator (TAG/TDG Group) Jwl Systems Department Worldlink Communications Pvt. Ltd. Jawalakhel, Nepal http://www.wlink.com.np http://teklimbu.wordpress.com BSD Networking, Microsoft Notworking
Re: [squid-users] storeDiskdSend OPEN: (35) Resource temporarily unavailable
On Mon, Jan 21, 2008, Monah Baki wrote: > Hi Tek, > > I am planning on moving 1500 users to the proxy, ofcourse the system > will change (hardware wise). Performance of squid should be > important, I knew diskd was still in "test" mode, but I did not > expect 4 users even though it was running for 27 days to display this > error messages. > For my own info, if I move to ufs, and if ufs is pretty stable, > performance wise is it close to diskd or better. Move it to AUFS, UFS isn't (currently) going to be very high performing. Diskd wasn't intended to be "testing"; it just turned out that the design made a couple of flawed assumptions about the rest of the Squid codebase. As always, I've got ideas on how to fix that; who knows when that'll happen. Adrian -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -
