[ https://issues.apache.org/jira/browse/CXF-3180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Willem Jiang resolved CXF-3180. ------------------------------- Resolution: Fixed Fix Version/s: 2.2.13 > Jaxrs/Jaxb unmarshaller code regularly causing performance problems & thread > lock-up > ------------------------------------------------------------------------------------ > > Key: CXF-3180 > URL: https://issues.apache.org/jira/browse/CXF-3180 > Project: CXF > Issue Type: Bug > Affects Versions: 2.2.6, 2.2.7, 2.2.8, 2.2.9, 2.2.10, 2.3.0, 2.2.11, > 2.2.12, 2.3.1 > Reporter: Willem Jiang > Assignee: Willem Jiang > Fix For: 2.3.2, 2.2.13, 2.4 > > > Please find below the details of the XML parsing performance bottleneck we > are seeing when processing REST POST requests with XML bodies. > The performance issue is caused by the fact that CXF code creates a new JAXB > Unmarshaller for every incoming request. The Unmarshaller is created quickly > enough but when it is used to do the unmarshalling, it needs to create a > Xerces SAXParser, which in turn calls DTDDVFactory.getInstance() and that > method is synchronized. When we try to process more than 200-300 requests per > second we start seeing a lot of threads blocked in DTDDVFactory.getInstance(). > The thread stack of one of the blocked threads is: > 1390532...@qtp-69728034-390" prio=10 tid=0x00002aab449c0000 nid=0x28f4 > waiting for monitor entry [0x000000004da26000] > java.lang.Thread.State: BLOCKED (on object monitor) > at > com.sun.org.apache.xerces.internal.impl.dv.DTDDVFactory.getInstance(DTDDVFactory.java:44) > waiting to lock <0x00002aaab279ed48> (a java.lang.Class for > com.sun.org.apache.xerces.internal.impl.dv.DTDDVFactory) > at > com.sun.org.apache.xerces.internal.parsers.XML11Configuration.<init>(XML11Configuration.java:538) > at > com.sun.org.apache.xerces.internal.parsers.XIncludeAwareParserConfiguration.<init>(XIncludeAwareParserConfiguration.java:125) > at > com.sun.org.apache.xerces.internal.parsers.XIncludeAwareParserConfiguration.<init>(XIncludeAwareParserConfiguration.java:86) > at sun.reflect.GeneratedConstructorAccessor4.newInstance(Unknown Source) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > at java.lang.reflect.Constructor.newInstance(Constructor.java:513) > at java.lang.Class.newInstance0(Class.java:355) > at java.lang.Class.newInstance(Class.java:308) > at > com.sun.org.apache.xerces.internal.parsers.ObjectFactory.newInstance(ObjectFactory.java:349) > at > com.sun.org.apache.xerces.internal.parsers.ObjectFactory.createObject(ObjectFactory.java:154) > at > com.sun.org.apache.xerces.internal.parsers.ObjectFactory.createObject(ObjectFactory.java:97) > at > com.sun.org.apache.xerces.internal.parsers.SAXParser.<init>(SAXParser.java:102) > at > com.sun.org.apache.xerces.internal.parsers.SAXParser.<init>(SAXParser.java:87) > at > com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.<init>(SAXParserImpl.java:332) > at > com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.<init>(SAXParserImpl.java:122) > at > com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl.newSAXParser(SAXParserFactoryImpl.java:76) > at > javax.xml.bind.helpers.AbstractUnmarshallerImpl.getXMLReader(AbstractUnmarshallerImpl.java:75) > at > javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:184) > at > javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:191) > at > org.apache.cxf.jaxrs.provider.JAXBElementProvider.unmarshalFromInputStream(JAXBElementProvider.java:216) > at > org.apache.cxf.jaxrs.provider.JAXBElementProvider.doUnmarshal(JAXBElementProvider.java:186) > at > org.apache.cxf.jaxrs.provider.JAXBElementProvider.readFrom(JAXBElementProvider.java:157) > at > org.apache.cxf.jaxrs.utils.JAXRSUtils.readFromMessageBody(JAXRSUtils.java:947) > at org.apache.cxf.jaxrs.utils.JAXRSUtils.processParameter(JAXRSUtils.java:560) > at > org.apache.cxf.jaxrs.utils.JAXRSUtils.processParameters(JAXRSUtils.java:525) > at > org.apache.cxf.jaxrs.interceptor.JAXRSInInterceptor.processRequest(JAXRSInInterceptor.java:230) > at > org.apache.cxf.jaxrs.interceptor.JAXRSInInterceptor.handleMessage(JAXRSInInterceptor.java:88) > at > org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:243) > locked <0x00002aab1e41a0e8> (a org.apache.cxf.phase.PhaseInterceptorChain) > at > org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:109) > at > org.apache.cxf.transport.http_jetty.JettyHTTPDestination.serviceRequest(JettyHTTPDestination.java:312) > ... > The application where the problem is seen defines an XSD file and uses > theJAXB compiler to create the data binding java class. It then creates a > "RESTResouce" java class that is CXF/JAXRS annotated and contains a method > that defines the processing of incoming POST requests. That method has one > argument, the JAXB-compiled class, which tells CXF to unmarshall the XML body > of the incoming request. > Just in case, you need more context, here is the top portion of the stack of > the thread that is inside the synchronized method: > "1840040...@qtp-69728034-391" prio=10 tid=0x00002aab44810000 nid=0x2dfa > waiting on condition [0x000000005be59000] > java.lang.Thread.State: RUNNABLE > at > org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.findLocalClass(DefaultClassLoader.java:211) > at > org.eclipse.osgi.internal.loader.BundleLoader.findLocalClass(BundleLoader.java:381) > at > org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:457) > at > org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:410) > at > org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:398) > at > org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:105) > at java.lang.ClassLoader.loadClass(ClassLoader.java:248) > at > org.eclipse.osgi.internal.loader.BundleLoader.loadClass(BundleLoader.java:326) > at > org.eclipse.osgi.framework.internal.core.BundleHost.loadClass(BundleHost.java:231) > at > org.eclipse.osgi.framework.internal.core.AbstractBundle.loadClass(AbstractBundle.java:1193) > at > org.springframework.osgi.util.BundleDelegatingClassLoader.findClass(BundleDelegatingClassLoader.java:99) > at > org.springframework.osgi.util.BundleDelegatingClassLoader.loadClass(BundleDelegatingClassLoader.java:156) > at java.lang.ClassLoader.loadClass(ClassLoader.java:248) > at > com.sun.org.apache.xerces.internal.impl.dv.ObjectFactory.findProviderClass(ObjectFactory.java:395) > at > com.sun.org.apache.xerces.internal.impl.dv.ObjectFactory.newInstance(ObjectFactory.java:350) > at > com.sun.org.apache.xerces.internal.impl.dv.DTDDVFactory.getInstance(DTDDVFactory.java:59) > locked <0x00002aaab279ed48> (a java.lang.Class for > com.sun.org.apache.xerces.internal.impl.dv.DTDDVFactory) > at > com.sun.org.apache.xerces.internal.impl.dv.DTDDVFactory.getInstance(DTDDVFactory.java:44) > locked <0x00002aaab279ed48> (a java.lang.Class for > com.sun.org.apache.xerces.internal.impl.dv.DTDDVFactory) > at > com.sun.org.apache.xerces.internal.parsers.XML11Configuration.<init>(XML11Configuration.java:538) > at > com.sun.org.apache.xerces.internal.parsers.XIncludeAwareParserConfiguration.<init>(XIncludeAwareParserConfiguration.java:125) > at > com.sun.org.apache.xerces.internal.parsers.XIncludeAwareParserConfiguration.<init>(XIncludeAwareParserConfiguration.java:86) > at sun.reflect.GeneratedConstructorAccessor4.newInstance(Unknown Source) > ... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.