On 3/7/06, Ryan Bowman <[EMAIL PROTECTED]> wrote: > This is not under load, but the poor performance is manifest with just one > user. > Oh, and by the way, this is now different hardware, a 32bit install, > with only 2GB RAM total, -Xmx1408m, but the problem is exists.
OK. > > uptime > 19:40 I was looking for the load average portion, but that's OK. Top has it. > > vmstat > procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu---- > r b swpd free buff cache si so bi bo in cs us sy id wa > 0 0 0 585448 91132 907528 0 0 2 6 67 28 0 0 100 > 0 Was this run while you were doing something with your client? I don't think we're going to see anything useful unless there is a constant load while you run these commands. This output show that exactly nothing is going on -- but that is correct if you aren't connecting with enough clients to actually see something in real-time. What I was hoping to see is that the last column (wa -- "wait") would have something other than 0. A value other than 0 would indicate that the CPU is waiting on I/O -- network or disk or something. > > netstat -s > Tcp: > 508 active connections openings > 172 passive connection openings > 0 failed connection attempts > 14 connection resets received > 83 connections established > 1476735 segments received > 1324213 segments send out > 60 segments retransmited > 0 bad segments received. > 7 resets sent > TcpExt: > 39 packets pruned from receive queue because of socket buffer overrun > ArpFilter: 0 > 411 TCP sockets finished time wait in fast timer > 1 time wait sockets recycled by time stamp > 70992 delayed acks sent > 4 delayed acks further delayed because of locked socket > Quick ack mode was activated 210 times > 18211 packets directly queued to recvmsg prequeue. > 3600 packets directly received from backlog > 42464 packets directly received from prequeue > 1072820 packets header predicted > 111 packets header predicted and directly queued to user > TCPPureAcks: 21212 > TCPHPAcks: 914601 > TCPRenoRecovery: 0 > TCPSackRecovery: 0 > TCPSACKReneging: 0 > TCPFACKReorder: 0 > TCPSACKReorder: 0 > TCPRenoReorder: 0 > TCPTSReorder: 0 > TCPFullUndo: 0 > TCPPartialUndo: 0 > TCPDSACKUndo: 0 > TCPLossUndo: 4 > TCPLoss: 0 > TCPLostRetransmit: 0 > TCPRenoFailures: 0 > TCPSackFailures: 0 > TCPLossFailures: 0 > TCPFastRetrans: 0 > TCPForwardRetrans: 0 > TCPSlowStartRetrans: 0 > TCPTimeouts: 25 > TCPRenoRecoveryFail: 0 > TCPSackRecoveryFail: 0 > TCPSchedulerFailed: 0 > TCPRcvCollapsed: 2339 > TCPDSACKOldSent: 210 > TCPDSACKOfoSent: 0 > TCPDSACKRecv: 0 > TCPDSACKOfoRecv: 0 > TCPAbortOnSyn: 0 > TCPAbortOnData: 3 > TCPAbortOnClose: 3 > TCPAbortOnMemory: 0 > TCPAbortOnTimeout: 6 > TCPAbortOnLinger: 0 > TCPAbortFailed: 0 > TCPMemoryPressures: 0 Looks OK. 60 retransmits isn't bad considering the uptime of this box. > > top -n 1 > top - 12:06:08 up 19:51, 5 users, load average: 0.02, 0.02, 0.06 > Tasks: 106 total, 1 running, 105 sleeping, 0 stopped, 0 zombie > Cpu(s): 0.2% us, 0.0% sy, 0.0% ni, 99.7% id, 0.0% wa, 0.0% hi, 0.0% si > Mem: 2074672k total, 1490552k used, 584120k free, 91132k buffers > Swap: 1052248k total, 0k used, 1052248k free, 907528k cached > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 12997 root 17 0 1788m 382m 89m S 2.0 18.9 3:17.09 java Again, we can't see anything measurable without some sustained client load while we look. BTW - You're brave to run jboss as root :-). > > netstat -tn | grep -v ESTABLISHED > Nothing. You would probably see something with a bigger client load, but, regardless, my theory is wrong about TCP TIME_WAITS being a problem if you have poor performance with a single client. Back to the drawing board :-(. Is it too late to rant about my feelings towards JBoss :-)? Would you be willing to deploy your EAR to Geronimo just to get another data point on this? If Geronimo is fast and JBoss is slow at least we can isolate it to a JBoss-on-Linux problem. I have no idea why it's working well on W2K3 and not Linux with this data. Can you add some logging to your app and timestamp your internal method calls to see how long JDBC queries are taking and other overhead. Find out which calls are taking so long -- the hot-spots. Also, add the following param to jboss' command-line -Dcom.sun.management.jmxremote -- you'll be able to connect to that JVM with a local instance of jconsole and watch threads, memory, and JMX instruments while the app runs. Also, also, you can run "kill -QUIT <jboss pid>" while the app is running to have a stacktrace of exactly what's being executed at that moment dumped to stdout. I'd like to see what's on your stack while one of your client requests is taking 5 seconds (though your timestamp logging will be more helpful than a raw stack anyway). -Bryan /* PLUG: http://plug.org, #utah on irc.freenode.net Unsubscribe: http://plug.org/mailman/options/plug Don't fear the penguin. */
