On 03/08/2013 12:06, Rainer Jung wrote:
> Overview:
>
> - one crash in tcnative during unit tests
> - MBeans indicate common loader is gone (is that expected?
> At least the common.loader entry still exists in catalina.properties)
Expected as StandardClassLoader has been removed. That decision is
reversible of there is a use case for having the class loader visible in
JMX.
> - some MBean data gone (resource cache statistics, environment content)
Resource cache is expected. Env is not. Might need to add more JMX stats
to new WebResource implementation.
> - Catalina:j2eeType=WebModule,name=//localhost attribute
> workDir switched from work/Catalina/localhost/_
> to work/Catalina/localhost/ROOT
Expected.
> - new SEVERE and Exceptions in unit test output, maybe expected
Unless the test fails then these should be expected.
> - bayeaux and tomcat-lite modules not bundled
Expected. I won't be copying these to tc8.0.x/trunk
> - MD5 OK
> - signatures OK
> - key in KEYS file
> - gz and zip for src and bin consistent
> ! src consistent with svn tag except:
> - modules bayeux and tomcat-lite not in src
See above.
> - file
> res/META-INF/tomcat-websocket.jar/services/javax.servlet.ServletContainerInitializer
> has Unix line ends in svn, but the src archives
> for Unix and Windows both contain it with DOS line ends
It wasn't treated as a text file. Now fixed.
> ! builds fine but
> - several warning about unsafe or unchecked operations.
> See full list at end of mail.
I was expecting all of those to be unfixable. To double-check I disabled
Eclipse's hide unavoidable generics errors and to my surprise ~30% were
easily fixable. These have been fixed. The rest do look to be unavoidable.
Digging into the 'fixes' all they appear to be doing is using a JRE
method that already has @SupressWarnings so arguably Eclipse was right.
Even so, I am no longer as strongly +1 for using this option as I have been.
> - build result looks consistent with binaries
> - no checkstyle complaints
> - only few Javadoc warnings, I fixed those
> ! Unit tests few failures and errors:
These need looking at. Note: Many of the tests are timing sensitive and
do not fail consistently. Making the tests less sensitive to timing is
an ongoing task.
>
> - Timing: TestWebSocketFrameClient.testConnectToServerEndpointSSL()
> took 63-66 seconds, but expects only 60 seconds.
>
>
> - On NIO in addition (same test):
>
> java.lang.NullPointerException
> at
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:276)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:692)
> at
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1592)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1550)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
>
>
> - On APR in addition a crash (core) in:
>
> b76dfe74 Java_org_apache_tomcat_jni_Socket_recv (30b528, b30ff188,
> b30fd09c, b29500, b30ff214, 0) + 34
> fbc0f780 * org/apache/tomcat/jni/Socket.recv(J[BII)I+32362
> fbc0f72c * org/apache/tomcat/jni/Socket.recv(J[BII)I+0
> fbc068ec *
> org/apache/coyote/http11/upgrade/AprServletInputStream.doRead(Z[BII)I+73
> (line 101)
> fbc068ec *
> org/apache/coyote/http11/upgrade/AbstractServletInputStream.read([BII)I+20
> (line 229)
> fbc068ec *
> org/apache/tomcat/websocket/server/WsFrameServer.onDataAvailable()V+46
> (line 89)
> fbc06b84 *
> org/apache/tomcat/websocket/server/WsHttpUpgradeHandler$WsReadListener.onDataAvailable()V+4
> (line 384)
> fbc0771c *
> org/apache/coyote/http11/upgrade/AbstractServletInputStream.onDataAvailable()V+11
> (line 337)
> fbc06b84 *
> org/apache/coyote/http11/upgrade/AbstractProcessor.upgradeDispatch(Lorg/apache/tomcat/util/net/SocketStatus;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+11
> (line 189)
> fbc073e0 *
> org/apache/coyote/AbstractProtocol$AbstractConnectionHandler.process(Lorg/apache/tomcat/util/net/SocketWrapper;Lorg/apache/tomcat/util/net/SocketStatus;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+171
> (line 1221)
> fbc06848 *
> org/apache/coyote/http11/Http11AprProtocol$Http11ConnectionHandler.process(Lorg/apache/tomcat/util/net/SocketWrapper;Lorg/apache/tomcat/util/net/SocketStatus;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+76
> (line 546)
> fbc073e0 *
> org/apache/tomcat/util/net/AprEndpoint$SocketProcessor.doRun()V+26 (line
> 4492)
> fbc06b84 *
> org/apache/tomcat/util/net/AprEndpoint$SocketProcessor.run()V+55 (line 4468)
> ...
>
> GDB tells my:
>
> #7 0xb76dfe74 in Java_org_apache_tomcat_jni_Socket_recv (e=0x30b528,
> o=0xb30ff188, sock=<optimized out>, buf=0xb30ff214, offset=0,
> toread=8192) at src/network.c:733
> 733 if ((ss = (*s->net->recv)(s->opaque, sb, &nbytes)) ==
> APR_SUCCESS)
>
> #7 0xb76dfe74 in Java_org_apache_tomcat_jni_Socket_recv (e=0x30b528,
> o=0xb30ff188, sock=<optimized out>, buf=0xb30ff214, offset=0,
> toread=8192) at src/network.c:733
> sb =
> "\000\006\026\200\000\fÞ̳\017Ñ\000·\205{Í·\205{س\017Ñ\234³\017Ñ\030³\017Ш·\227(p»\204H\200ò\032\226x\000\000\000\023\000\000\000\000ûÁ\227hò\032\227\020³\017Ñ8³\017Ñ8ûÀhH\000\000\b¹³\017Ñh³\017ÑHûÀhH³\017ÑP\000\000\002°·\211+Ø\000\000\000u\000\000\000\000ò\032\227
> ò\032\226\210ò\032\226\210\000\000\000u\000\000\000\000³\017ÑhûÀhH»\204Hh¼Ð¬h\000\000\001\032\000\000\000\000³\017Ñ\230·\225x'·\225xH³\017Ò$³\017Ñ
>
> ³\017Ñ8·\225}ø\000\060´\000\000\000\000\000\000\000\000¶\000\000\000\000ûÁÄÀ"...
> s = 0xb29500
> nbytes = 8192
> ss = <optimized out>
>
> (gdb) print *s
> $5 = {pool = 0xb294c0, child = 0x0, sock = 0x0, opaque = 0xb297d8,
> jsbbuff = 0x330398 "HTTP/1.1 101 Switching Protocols\r\nServer:
> Apache-Coyote/1.1\r\nUpgrade: websocket\r\nConnection:
> upgrade\r\nSec-WebSocket-Accept: tPEj68f24a5073zco+NHsEUCKpY=\r\nDate:
> Fri, 02 Aug 2013 12:52:53 GMT\r\n\r\n",
> jrbbuff = 0xbfbc78 "GET /firehose HTTP/1.1\r\nSec-WebSocket-Version:
> 13\r\nUpgrade: websocket\r\nHost: localhost:59353\r\nSec-WebSocket-Key:
> qzlX6/aQxs4TAKyKCb+XLg==\r\nConnection: upgrade\r\n\r\n", net = 0x0, pe
> = 0xb87958, last_active = 0, timeout = -2}
>
> The reason for the crash is likely:
>
> (gdb) print s->net
> $7 = (tcn_nlayer_t *) 0x0
>
> So dereferencing it in (*s->net->recv) gives a segfault.
>
>
> - TestWsWebSocketContainer:
>
> Testcase: testMaxMessageSize03 took 0.426 sec
> Caused an ERROR
> The WebSocket session has been closed and no method (apart from close())
> may be called on a closed session
> java.lang.IllegalStateException: The WebSocket session has been closed
> and no method (apart from close()) may be called on a closed session
> at
> org.apache.tomcat.websocket.WsSession.checkState(WsSession.java:607)
> at
> org.apache.tomcat.websocket.WsSession.getUserProperties(WsSession.java:556)
> at
> org.apache.tomcat.websocket.WsRemoteEndpointImplBase.getBlockingSendTimeout(WsRemoteEndpointImplBase.java:360)
> at
> org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendPartialString(WsRemoteEndpointImplBase.java:195)
> at
> org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendString(WsRemoteEndpointImplBase.java:155)
> at
> org.apache.tomcat.websocket.WsRemoteEndpointBasic.sendText(WsRemoteEndpointBasic.java:37)
> at
> org.apache.tomcat.websocket.TestWsWebSocketContainer.doMaxMessageSize(TestWsWebSocketContainer.java:859)
> at
> org.apache.tomcat.websocket.TestWsWebSocketContainer.testMaxMessageSize03(TestWsWebSocketContainer.java:830)
>
>
> - TestWebappClassLoaderExecutorMemoryLeak NIO:
>
> Testcase: testTimerThreadLeak took 4.932 sec
> FAILED
> null
> junit.framework.AssertionFailedError
> at
> org.apache.catalina.loader.TestWebappClassLoaderExecutorMemoryLeak.testTimerThreadLeak(TestWebappClassLoaderExecutorMemoryLeak.java:72)
>
>
>
> - JMX MBean-Comparison with TC 7:
>
> - MBean Catalina:type=ServerClassLoader,name=common no longer exists
Expected.
> - MBeans j2eeType=WebModule attribute
> parentClassLoader is no longer of type
> org.apache.catalina.loader.StandardClassLoader
> but instead java.net.URLClassLoader
Expected.
> - MBeans j2eeType=Servlet and j2eeType=WebModule no longer
> have attributes eventProvider and statisticsProvider
Expected. They were hard-coded to false so were removed.
> - MBeans j2eeType=WebModule no longer have attributes
> saveConfig, staticResources and mappingObject
saveConfig - removed as was unused
staticResoucres - new WebResources impl
mappingObject - removed (unnecessary duplicate)
> - MBeans type=Cache replaced by type=WebResourceRoot
> - old attributes accessCount, hitsCount, desiredEntryAccessRatio,
> cacheSize, maxAllocateIterations removed
> - new attributes cacheMaxObjectSize, cacheTtl, cachingAllowed,
> allowLinking, stateName, objectName, domain added
All expected. Attributes can be added to new impl as required.
> - MBean type=NamingResourcesno longer provides the arrays
> resources, environments and resourceLinks and attribute container.
> modelerType changed from
> org.apache.catalina.mbeans.NamingResourcesMBean
> to org.apache.tomcat.util.modeler.BaseModelMBean
That is not expected. Need to look into that.
> - MBeans type=Loader no longer have attributes repositories,
> repositoriesString and searchExternalFirst.
> Probably replaced by the new Resources impl
Correct.
> - MBeans type=WebappClassLoader no longer has attribute
> searchExternalFirst.
> Probably replaced by the new Resources impl.
Corrct.
> - ThreadPool, RequestProcessor and GlobalRequestProcessor now
> for nio instead of bio as default; corresponding name changes
> in thread names, class names etc. and increased maxConnections
Expected.
> - nio based ThreadPools switch useComet, usePolling and
> useCometTimeout from "false" to "true", for http also
> useSendfile switched from "false" to "true".
> New attributes keepAliveCount, pollerThreadCount,
> pollerThreadPriority, selectorTimeout, selectorPool and
> oomParachute
Expected.
> - nio based ProtocolHandler new attributes connectionCount,
> cometSupported, cometTimeoutSupported and sendfileSupported.
> For http in addition pollerThreadCount, pollerThreadPriority,
> acceptorThreadPriority, useSendfile, endpoint and selectorTimeout.
> Attribute disableKeepAlivePercentage only existed for http and
> is gone there now.
Expected.
> - Example Servlets wsChat, wsEchoMessage, wsEchoStream, wsSnake
> removed
Need to look in to JMX visibility of new WebSocket impl.
> - Filter o.a.t.websocket.server.WsFilter added to all 5 webapps
> (ROOT, docs, examples, manager, host-manager)
Expected.
> - order of jstl.jar and standard.jar in loader for /examples
> switched
Odd but should not be an issue.
> - MBean type=Environment now className
> org.apache.tomcat.util.descriptor.web.ContextEnvironment
> instead of org.apache.catalina.deploy.ContextEnvironment
Expected.
> - type=Mapper singleton, no longer per connector
Expected.
> - Catalina:type=Valve,host=localhost,name=AccessLogValve
> locale changed from en_US to en, fileDateFormat changed
> from yyyy-MM-dd to .yyyy-MM-dd, prefix changed from
> localhost_access_log. to localhost_access_log (prefix and
> fileDateFormat changes OK), resolveHosts attribute gone (unused),
> info attribute gone (not a big loss)
Expected.
> - Catalina:type=Valve,host=localhost,name=ErrorReportValve
> attribute info gone (not a big loss)
Expected.
> - Catalina:type=Engine attribute baseDir replaced by catalinaBase
Expected.
> - Catalina:j2eeType=WebModule,name=//localhost attribute
> ignoreAnnotations switched from "true" to "false",
Need to look in to that.
> attribute workDir switched from work/Catalina/localhost/_
> to work/Catalina/localhost/ROOT
Expected.
> - Catalina:type=Deployer,host=localhost attribute
> configClass: org.apache.catalina.startup.ContextConfig gone
Expected.
> - new (platform) MBeans java.nio:type=BufferPool,name=direct
> and java.nio:type=BufferPool,name=mapped
OK.
> - platform MBean type=Logging: LoggerNames javax, javax.management and
> javax.management.mbeanserver removed, org.apache.tomcat.websocket
> and org.apache.tomcat.websocket.WsWebSocketContainer added
OK.
> - platform MBean java.lang:type=Threading shows 4 more (daemon)
> threads
OK.
> - platform MBean java.lang:type=OperatingSystem show 88 open file
> descriptors instead of 61
Worth just checking what those files are.
> - platform MBean java.lang:type=Runtime shows 3 system properties
> *.jarsToSkip replaced by t.u.s.StandardJarScanFilter.jarsToScan and
> t.u.s.StandardJarScanFilter.jarsToSkip
Expected.
> Build and tests were done using Java 1.7.0_25. OS was Solaris 10 Sparc,
> tcnative was 1.1.27 based on APR 1.4.8 and OpenSSL 1.0.1e (plus a few
> patches).
>
> Unit test warnings (very similar to 7.0.35 and before, but two new ones,
> one NPE in access log, one testResources warning):
>
> - org.apache.tomcat.util.descriptor.web.TestWebXmlOrdering
> - (old) BIO, NIO and APR: [main]
> org.apache.tomcat.util.descriptor.web.WebXml.orderWebFragments
> Used a wrong fragment name z at web.xml absolute-ordering tag!
> - (new) BIO, NIO and APR: [main]
> org.apache.tomcat.util.descriptor.web.WebXml.setVersion
> Unknown version string [0.0]. Default version will be used.
Look OK.
> - org.apache.tomcat.util.net.TestCustomSsl
> - (old) BIO twice "Exception getting SSL attributes"
> in org.apache.coyote.http11.Http11Processor actionInternal
> exception is:
> javax.net.ssl.SSLProtocolException: handshake alert: no_certificate
> - (old) NIO twice "Exception re-negotiating SSL connection"
> in org.apache.coyote.http11.Http11NioProcessor actionInternal
Need to check.
> - org.apache.catalina.startup.TestHostConfigAutomaticDeployment
> - (new) BIO, NIO and APR: [main] 3 times each
> org.apache.catalina.startup.HostConfig deployDescriptor
> A docBase .../webapps/myapp inside the host appBase has been
> specified, and will be ignored
> - (new) BIO, NIO and APR: [main] 2 times each
> org.apache.catalina.startup.HostConfig deployWARs
> The directory [.../webapps/myapp] will be ignored because the WAR
> [.../webapps/myapp.war] takes priority and unpackWARs is false
Expected.
> - tribes: several warnings (41), numbers up by 22 from 7.0.42.
> Mostly same types of messages.
Probably OK but should check.
> Unit test SEVERE messages: 439 total (+218 rel TC 7).
That seems rather high.
> About 200 of those are of types not observed for TC 7:
>
> Count Message
>
> 48 : ContainerBase.addChild: start:
Need to look into those.
> 48 : Error deploying configuration descriptor
> .../conf/Tomcat/localhost/myapp.xml
Related to the above.
> 12 : Error listenerStart
Need to look into these.
> 12 : For security constraints with URL pattern [/] only the HTTP
> methods [POST] are covered. All other methods are uncovered.
> 6 : For security constraints with URL pattern [/] only the HTTP
> methods [GET] are covered. All other methods are uncovered.
> 6 : For security constraints with URL pattern [/test] only the HTTP
> methods [POST] are covered. All other methods are uncovered.
> 3 : For security constraints with URL pattern [/test] only the HTTP
> methods [PUT] are covered. All other methods are uncovered.
> 3 [main]
> org.apache.tomcat.util.descriptor.web.SecurityConstraint.findUncoveredHttpMethods
> For security constraints with URL pattern [/test] only the HTTP methods
> [GET] are covered. All other methods are uncovered.
> 3 [main]
> org.apache.tomcat.util.descriptor.web.SecurityConstraint.findUncoveredHttpMethods
> For security constraints with URL pattern [/test] only the HTTP methods
> [POST] are covered. All other methods are uncovered.
> 3 [main]
> org.apache.tomcat.util.descriptor.web.SecurityConstraint.findUncoveredHttpMethods
> For security constraints with URL pattern [/test] the HTTP methods [GET]
> are uncovered.
> 3 [main]
> org.apache.tomcat.util.descriptor.web.SecurityConstraint.findUncoveredHttpMethods
> For security constraints with URL pattern [/test] the HTTP methods
> [POST] are uncovered.
All expected. These are new tests for the new 3.1 uncovered methods feature.
> 9 : Error reading request, ignored
> 9 : Exception fixing docBase for context [/myapp]
> 6 [localhost-startStop-1]
> org.apache.catalina.core.StandardContext.listenerStart Exception sending
> context initialized event to listener instance of class
> websocket.echo.WsConfigListener
> 6 [localhost-startStop-1]
> org.apache.catalina.core.StandardContext.startInternal Context
> [/examples] startup failed due to previous errors
> 6 [localhost-startStop-1]
> org.apache.catalina.core.StandardContext.startInternal Error listenerStart
> 6 : Context [/examples] startup failed due to previous errors
> 6 : Context [/myapp] startup failed due to previous errors
> 6 : Exception sending context initialized event to listener instance
> of class websocket.echo.WsConfigListener
> 6 : Skipped installing application listeners due to previous error(s)
> 3 : No error handling configured for
> [org.apache.tomcat.websocket.pojo.TestPojoEndpointBase$Bug54716] and the
> following error occurred
> 1 : Error configuring application listener of class
> org.apache.tomcat.util.descriptor.web.ApplicationListener@12cc06
> 1 : Error configuring application listener of class
> org.apache.tomcat.util.descriptor.web.ApplicationListener@1605b09
> 1 : Error configuring application listener of class
> org.apache.tomcat.util.descriptor.web.ApplicationListener@17024fc
> 1 : Error configuring application listener of class
> org.apache.tomcat.util.descriptor.web.ApplicationListener@18896b4
> 1 : Error configuring application listener of class
> org.apache.tomcat.util.descriptor.web.ApplicationListener@4114a7
> 1 : Error configuring application listener of class
> org.apache.tomcat.util.descriptor.web.ApplicationListener@f4b327
The above error message needs fixing.
The above group of messages needs checking.
> The rest looks very similar to TC 7:
>
> Most of those are of type "Servlet.service() for servlet ... threw
> exception" (137 times, unchanged).
>
> Second biggest block (unchanged):
>
> Count message
> 27 [junit] TIMESTAMP SEVERE [main]
> org.apache.tomcat.util.digester.Digester.startElement Begin event threw
> exception
> 3 [junit] TIMESTAMP SEVERE [main]
> org.apache.tomcat.util.digester.Digester.endElement End event threw
> exception
>
> Third biggest block is 15 (-4) messages from the mem leak detection
> during context unload (same types as before), but in addition one new
> message
>
> The web application [] is still processing a request that has yet to
> finish. This is very likely to create a memory leak. You can control the
> time allowed for requests to finish by using the unloadDelay attribute
> of the standard Context implementation.
>
> in org.apache.catalina.connector.TestMaxConnections for BIO, NIO, APR 2
> times each.
>
>
> "Exception" in output of unit tests: 475 times (+256 rel. TC 7).
>
> Top exception counts (some new entries):
Should be covered by looking at the warnings as per above.
> "unchecked" warnings during build:
See above.
Many thanks for such a thorough analysis.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
