fd patch to squid

2006-07-19 Thread Martin Stransky

Hello guys,

As you probably know, we have one RH specific patch to squid. It's for 
dynamic configuration of filedescriptors number, via. squid.conf. I 
reviewed it for the 2.6 branch, fixed some bugs (which comes with squid 
 reconfiguration) so I think it's stable enough now.


If you find it useful, put it please to http://devel.squid-cache.org/ or 
I can make place for it on http://people.redhat.com/stransky/squid/, 
it's up to you.


Regards,

Martin
--- squid-2.6.STABLE1/configure.fd	2006-07-18 10:09:15.0 +0200
+++ squid-2.6.STABLE1/configure	2006-07-18 10:09:15.0 +0200
@@ -2970,6 +2970,34 @@
   AMDEP_FALSE=
 fi
 
+if false; then
+  USE_FD_CONFIG_TRUE=
+  USE_FD_CONFIG_FALSE='#'
+else
+  USE_FD_CONFIG_TRUE='#'
+  USE_FD_CONFIG_FALSE=
+fi
+# Check whether --enable-fd-config or --disable-fd-config was given.
+if test ${enable_fd_config+set} = set; then
+  enableval=$enable_fd_config
+   if test $enableval = yes ; then
+echo File descriptor config enabled
+cat  confdefs.h \EOF
+#define FD_CONFIG 1
+EOF
+
+
+
+if true; then
+  USE_FD_CONFIG_TRUE=
+  USE_FD_CONFIG_FALSE='#'
+else
+  USE_FD_CONFIG_TRUE='#'
+  USE_FD_CONFIG_FALSE=
+fi
+  fi
+
+fi
 
 
 
--- squid-2.6.STABLE1/include/autoconf.h.in.fd	2006-07-18 10:09:15.0 +0200
+++ squid-2.6.STABLE1/include/autoconf.h.in	2006-07-18 10:09:15.0 +0200
@@ -36,6 +36,9 @@
 /* Traffic management via delay pools. */
 #undef DELAY_POOLS
 
+/* Filedesc managment */
+#undef FD_CONFIG
+
 /* Enable following X-Forwarded-For headers */
 #undef FOLLOW_X_FORWARDED_FOR
 
--- squid-2.6.STABLE1/configure.in.fd	2006-07-18 10:09:15.0 +0200
+++ squid-2.6.STABLE1/configure.in	2006-07-18 10:09:15.0 +0200
@@ -501,6 +501,16 @@
   fi
 ])
 
+AM_CONDITIONAL(USE_FD_CONFIG, false)
+AC_ARG_ENABLE(fd-config,
+[  --enable-fd-configEnable filedesc config to configure maximal number of used filedescriptors],
+[ if test $enableval = yes ; then
+echo Filedesc config enabled
+AC_DEFINE(FD_CONFIG)
+AM_CONDITIONAL(USE_FD_CONFIG, true)
+  fi
+])
+
 dnl This is a developer only option. Developers know how to set defines
 dnl
 dnl AC_ARG_ENABLE(mem-gen-trace,
--- squid-2.6.STABLE1/src/fd.c.fd	2006-06-11 19:06:25.0 +0200
+++ squid-2.6.STABLE1/src/fd.c	2006-07-18 10:09:15.0 +0200
@@ -61,7 +61,7 @@
 {
 if (fd  Biggest_FD)
 	return;
-assert(fd  Squid_MaxFD);
+assert(fd  SQUID_NUMFD);
 if (fd  Biggest_FD) {
 	/*
 	 * assert that we are not closing a FD bigger than
@@ -206,7 +206,7 @@
 {
 int i;
 fde *F;
-for (i = 0; i  Squid_MaxFD; i++) {
+for (i = 0; i  SQUID_NUMFD; i++) {
 	F = fd_table[i];
 	if (!F-flags.open)
 	continue;
@@ -223,7 +223,7 @@
 int
 fdNFree(void)
 {
-return Squid_MaxFD - Number_FD - Opening_FD;
+return SQUID_NUMFD - Number_FD - Opening_FD;
 }
 
 int
@@ -252,10 +252,10 @@
 /*
  * Calculate a new reserve, based on current usage and a small extra
  */
-new = Squid_MaxFD - Number_FD + XMIN(25, Squid_MaxFD / 16);
+new = SQUID_NUMFD - Number_FD + XMIN(25, SQUID_NUMFD / 16);
 if (new = RESERVED_FD)
 	return;
-x = Squid_MaxFD - 20 - XMIN(25, Squid_MaxFD / 16);
+x = SQUID_NUMFD - 20 - XMIN(25, SQUID_NUMFD / 16);
 if (new  x) {
 	/* perhaps this should be fatal()? -DW */
 	debug(51, 0) (WARNING: This machine has a serious shortage of filedescriptors.\n);
--- squid-2.6.STABLE1/src/mem.c.fd	2006-05-20 23:51:49.0 +0200
+++ squid-2.6.STABLE1/src/mem.c	2006-07-18 10:09:15.0 +0200
@@ -335,13 +335,13 @@
 memDataInit(MEM_HTTP_HDR_CONTENT_RANGE, HttpHdrContRange, sizeof(HttpHdrContRange), 0);
 memDataInit(MEM_INTLIST, intlist, sizeof(intlist), 0);
 memDataInit(MEM_MEMOBJECT, MemObject, sizeof(MemObject),
-	Squid_MaxFD  3);
+	SQUID_NUMFD  3);
 memDataInit(MEM_MEM_NODE, mem_node, sizeof(mem_node), 0);
 memDataInit(MEM_NETDBENTRY, netdbEntry, sizeof(netdbEntry), 0);
 memDataInit(MEM_NET_DB_NAME, net_db_name, sizeof(net_db_name), 0);
 memDataInit(MEM_RELIST, relist, sizeof(relist), 0);
 memDataInit(MEM_REQUEST_T, request_t, sizeof(request_t),
-	Squid_MaxFD  3);
+	SQUID_NUMFD  3);
 memDataInit(MEM_STOREENTRY, StoreEntry, sizeof(StoreEntry), 0);
 memDataInit(MEM_WORDLIST, wordlist, sizeof(wordlist), 0);
 memDataInit(MEM_CLIENT_INFO, ClientInfo, sizeof(ClientInfo), 0);
--- squid-2.6.STABLE1/src/cf.data.pre.fd	2006-07-18 10:09:15.0 +0200
+++ squid-2.6.STABLE1/src/cf.data.pre	2006-07-18 10:09:15.0 +0200
@@ -5027,4 +5027,16 @@
 	or response to be rejected.
 DOC_END
 
+NAME: max_filedesc
+IFDEF: FD_CONFIG
+TYPE: int
+DEFAULT: 1024
+LOC: Config.max_filedesc
+DOC_START
+	The maximum number of open file descriptors.
+
+	WARNING: Changes of this value isn't respected by reconfigure 
+	command. This value should be changed only if there isn't 
+	any active squid process.
+DOC_END
 EOF
--- squid-2.6.STABLE1/src/ipc.c.fd	2006-06-24 11:41:36.0 +0200
+++ 

Re: fd patch to squid

2006-07-19 Thread Henrik Nordstrom
ons 2006-07-19 klockan 15:13 +0200 skrev Martin Stransky:
 Hello guys,
 
 As you probably know, we have one RH specific patch to squid.

Yes, found this out by accident some time ago..

  It's for 
 dynamic configuration of filedescriptors number, via. squid.conf. I 
 reviewed it for the 2.6 branch, fixed some bugs (which comes with squid 
   reconfiguration) so I think it's stable enough now.

A quick review of the patch indicates it's a bit overcomplicated

And I'm not that convinced you really need it either.. A simple
directive limiting Squid_MaxFD and a high limit specified to configure
would do nearly as good.

The parts eleminating SQUID_MAXFD constant dependencies is good.

Note: The patch does not account for comm_select_win32.c, causing
problems for the win32 plaform..

The parts messing with comm_select.c doesn't feel warranted. Systems
using select() is generally not good at handling more than FD_SETSIZE
connections anyway, and the select interface as such crumbles a bit when
used for many concurrent connections. Also not that keen on using a
custom type for the fd_set given to select(), even if we already make
very strict assumptions about fd_set being a bit array.. (but I have
heard of some systems using an array of ints...)

The bit array stuff doesn't feel needed, as it's only used by the
changes to comm_select.c.

Renaming Squid_MaxFD to SQUID_NUMFD does not feel right. Our naming
convention uses all caps for preprocessor constants only, and even with
the patch it's still the max limit this Squid can handle after startup..
(same as Squid_MaxFD).


What I'd like to see for doing this proper is

* The stuff eleminating SQUID_MAXFD dependencies in arrays etc.

* The directive for setting the fd limit

* Checks in comm_select_init or similar to lower Squid_MaxFD if it's
above what the select loop implementation can handle. This means
comm_select.c and comm_select_win32.c limiting Squid_MaxFD to
FD_SETSIZE. (eleminating the similar check from main.c). Some thinking
needed to get rid of comm_select dependencies.. (probably a new
comm_select_fdlimit call needed before comm_init)

* Moving the FD_SETSIZE override from squid.h to comm_select.c before
include of squid.h..

* After this SQUID_MAXFD should only be referenced from the FD_SETSIZE
override in comm_select.c.

Regards
Henrik


signature.asc
Description: Detta är en digitalt signerad	meddelandedel


2.6.STABLE2 approaching

2006-07-19 Thread Henrik Nordstrom
The 2.6.STABLE2 release is approaching with many critical bug fixes.

but Bug #1677 kind of a blocker for the release.. not due to it being a
significant Squid bug, but due to it causing some broken web sites to
fail which worked in 2.5..

Regards
Henrik



signature.asc
Description: Detta är en digitalt signerad	meddelandedel


Re: 2.6.STABLE2 approaching

2006-07-19 Thread Guido Serassio

Hi Henrik,

At 18.25 19/07/2006, Henrik Nordstrom wrote:


The 2.6.STABLE2 release is approaching with many critical bug fixes.

but Bug #1677 kind of a blocker for the release.. not due to it being a
significant Squid bug, but due to it causing some broken web sites to
fail which worked in 2.5..


I think that Bug #1681 should be also investigated before STABLE2.
The problem is happening on different platforms with the same symptom 
with too much frequency.


I will try to do more investigation during the next weekend.

Regards

Guido



-

Guido Serassio
Acme Consulting S.r.l. - Microsoft Certified Partner
Via Lucia Savarino, 1   10098 - Rivoli (TO) - ITALY
Tel. : +39.011.9530135  Fax. : +39.011.9781115
Email: [EMAIL PROTECTED]
WWW: http://www.acmeconsulting.it/