>Also I wrote when stall happens I can't telnet to port 80 on web server >host - which means it is not just program causing stall. Are you trying this from the same host as the benchmark program? I wonder if a 2nd host would have the same problem.
-----Original Message----- From: Peter Zaitsev [mailto:[EMAIL PROTECTED] Sent: Monday, October 31, 2005 3:53 PM To: support@pfsense.com Subject: Re: [pfSense Support] Network Device pooling On Mon, 2005-10-31 at 16:31 -0500, Scott Ullrich wrote: > Are we absolutely sure this program works as intended? Personally I > wouldn't trust anything like this but smartbits. Well... It works if filtering is disabled on pfsese - this is what worries me. If the program would be broken it should not work in both cases. Also I wrote when stall happens I can't telnet to port 80 on web server host - which means it is not just program causing stall. If it is protection on FreeBSD side from too much activity from same IP (Ie as it limits response to flood ping) this would be good to know. I hope this problem is actually something like that - I know there are a lot of FreeBSD based routers out where - if it would be broken for real workloads something would scream already. One more interesting thing I noticed: Percentage of the requests served within a certain time (ms) 50% 32 66% 33 75% 33 80% 33 90% 44 95% 295 98% 324 99% 330 100% 21285 (longest request) Even if apache benchmark does not timeout it often shows too long response rate - (21 sec in this case) What I've noticed - it can be 3, 9 or 21 secs in this case - This really look like the times at which SYN packets are resent by TCP/IP stacks if no reply for previous one arrives. Doing more experiments I also discovered I can increase chance of passing benchmark (still not to 100%) if i reduce tcp_fin_timeout and increase ip_local_port_range variables ob my test driver host. This still brings the question why with filtering and without behavior is different but it makes me worry less :) > > Scott > > > On 10/31/05, Peter Zaitsev <[EMAIL PROTECTED]> wrote: > > On Mon, 2005-10-31 at 16:25 -0500, Scott Ullrich wrote: > > > >apr_poll: The timeout specified has expired (70007) > > > > > > What is the above from? Your benchmark testing box? > > > > Yes. This is output from apache benchmark program. > > > > > > Benchmarking 111.111.111.158 (be patient) > > Completed 10000 requests > > Completed 20000 requests > > Completed 30000 requests > > apr_poll: The timeout specified has expired (70007) > > Total of 30517 requests completed > > > > > > > > > > > > On 10/31/05, Peter Zaitsev <[EMAIL PROTECTED]> wrote: > > > > On Mon, 2005-10-31 at 15:48 -0500, Scott Ullrich wrote: > > > > > Are you viewing the traffic queue status? This would be normal if you are... > > > > > > > > Heh, > > > > > > > > yes good quess. These were running in the other window. > > > > > > > > > > > > So here is the output for "stalled" case > > > > > > > > # pfctl -ss | wc -l > > > > 51898 > > > > > > > > I have number of states set to 100.000 in advanced page so it is not > > > > peak number. > > > > > > > > > > > > Note what really surprises me is the number of request when if fails: > > > > > > > > apr_poll: The timeout specified has expired (70007) > > > > Total of 28217 requests completed > > > > > > > > This number of 28217 is seen so often... Sometimes it is a bit more ot > > > > less but it is very frequently withing +/- 100 of it. > > > > > > > > I was asked if I can connect to the remote box when this problem happens > > > > - yes. I can SSH to the same box which runs Apache, but I can't > > > > connect to the port 80 when this problem happens. > > > > > > > > So it looks like it does not like to see all these states corresponding > > > > to the same target port number. > > > > > > > > > > > > > > > > > > > > > > Scott > > > > > > > > > > > > > > > On 10/31/05, Peter Zaitsev <[EMAIL PROTECTED]> wrote: > > > > > > On Mon, 2005-10-31 at 14:39 -0500, Scott Ullrich wrote: > > > > > > > On 10/31/05, Fleming, John (ZeroChaos) <[EMAIL PROTECTED]> wrote: > > > > > > > > I wonder if part of the problem is PF isn't seeing the TCP tear down. It > > > > > > > > seems a little odd that the max gets hit and nothing else gets through. > > > > > > > > I guess it could be the benchmark isn't shutting down the session right > > > > > > > > after its down transferring data, but I would think it would kill the > > > > > > > > benchmark client to have 10K(ish) of open TCP sessions. > > > > > > > > > > > > > > One way to deterimine this would be to run pfctl -ss | wc -l once > > > > > > > pfSense stops responding? > > > > > > > > > > > > Very interesting.... > > > > > > > > > > > > I tried running this before the problems but it looks strange already: > > > > > > > > > > > > # pfctl -ss | wc -l > > > > > > 4893 > > > > > > Killed > > > > > > # pfctl -ss | wc -l > > > > > > 23245 > > > > > > Killed > > > > > > > > > > > > There is nothing in dmesg or system logs. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
