Thanks for all the suggestions. Serge wrote: > I've seen something like this, and we were able to diagnose it on a Unix > environment by using a program called truss > (http://www.scit.wlv.ac.uk/cgi-bin/mansec?1+truss) which allows you to > watch every file that is getting read. I don't know of a Windows > equivalent. Most Java profilers I know of do not help you find IO > related issues very well, so you might want to look for a lower level OS > tool to see what's changing. > > What we found was that the application was looking for a class that did > not exist. Java does not hold a cache entry for "class not found", so > it kept opening every jar in the entire classpath to find said class, > numerous times for a single http request because of where this library > was used. You might want to check all the resin/lib and WEB-INF/lib and > whatever other classes to see if a non-critical class is missing but > being searched for over and over. I have watched file access using FileMon on Windows. Unfortunately, I don't get enough information to see if I have the same problem you had. But since the inital start of the application is also slow, I lean towards there is a generic problem with class loading and/or dependency checking. There seem to occur several depenency checks during a single request, which is why a simple page can take 40 seconds. If I increase the dependency check intervall, so that there is only one check during the request monitored by FileMon, it will finish much faster.
Scott wrote: > Do you see a difference if you remove the resin.dll? (Which would > disable the JNI.) > Not much. > Also, what is the stack trace for the slow read() call? With JNI, 57% of the time for loading a single page (total 78 s) is spent in com.caucho.vfs.JniStream.read cased by (only!!!) 22 calls from com.caucho.vfs.ReadStream.readBuffer com.caucho.vfs.ReadStream.waitForRead com.caucho.server.port.TcpConnection.run ... Without JNI 43% of the time (total 33 s) is spent in java.io.File.lastModified caused by 28000 calls (depency checking over and over) from com.caucho.vfs.FilePath.getLastModified com.caucho.vfs.Depend.isModified ... and 41% of the time is spent in java.io.File.length called 28000 times from com.caucho.vfs.FilePath.getLength com.caucho.vfs.Depend.isModified ... I created a simple test page, calling File.length and File.lastModified 100'000 times and measured the time. I then ran the page in another webapp, that does not seem to be affected (on the same computer though). The results were the same (about 0.2 ms / operation - not sure if this is normal though; I'll have to compare to the office computer next week). Anyway, this could indicate the problem is not with the I/O calls themselves, but the number of I/O calls... (Which makes Serges tip more interesting.) Again, I will have to profile at the other computer next week and compare. Tom Hintz wrote: > I’ve seen small TCP/IP MTU sizes cause similar behavior. We had VPN > software that set the MTU to less than the IP default, which I think > should have been 1500(?). Result was packet fragmentation that caused > huge delays in the Microsoft TCP implementation. I think the MTU > setting, in this case, was in the VPN software. I have not run any VPN software other than the built in PPTP client, and have not changed the MTU of the NICs. (MTU isn't fetched via DHCP is it?) I thought maybe there was a problem fetching external DTDs when reading Hibernate mappings or whatever (lots of profiling time spent parsing doc types and my ISP has had some issues lately). However, inactivating the NIC altogether makes no difference. > By the way, the CPU usage is peaking all the way through. Around 49% on > my dual core system. > (And there is even more memory on the computer with the problem than the > other one) > > >> -----Original Message----- >> From: [EMAIL PROTECTED] >> [mailto:[EMAIL PROTECTED] Behalf Of Mattias Jiderhamn >> Sent: Friday 05 October 2007 11:20 >> To: Resin >> Subject: [Resin-interest] Slooow file reads (really weird!) >> >> >> Hi list. >> I have my J2EE webapp on an external hard drive, which I carry between >> my office and my home computer. >> On each computer - running Windows XP and Java 1.5 - I have a Resin >> (3.0.22) installation and a shortcut to start Resin with the server root >> on the external drive. >> This has worked flawlessly for over a year. >> >> Now suddenly (ok, after returning from vacation), the application is >> immensely slow - on one of the computers! >> Starting Resin and the application now takes anywhere from 3 to 5 >> minutes on my home computer, compared to the usual 30 or so seconds. >> Loading a simple page can take 40 seconds. It seems that most of the >> time is spent inside the disk access of the dependency checking. >> (Turning dependency checking off was much faster, but still slower than >> normal). >> I tried to copy the project to the internal drive to see if there was >> some interface hardware issue - no difference. >> I profiled the application with JProfiler, and it thinks there is a >> hotspot in com.caucho.vsf.JniStream.read(). Why...? >> >> I am running out of ideas on what could be wrong and how to track it down. >> >> Any tips would be much welcome! >> >> /Mattias _______________________________________________ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
