DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT <http://nagoya.apache.org/bugzilla/show_bug.cgi?id=17630>. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE.
http://nagoya.apache.org/bugzilla/show_bug.cgi?id=17630 Java extension method cache keys are not unique Summary: Java extension method cache keys are not unique Product: XalanJ2 Version: CurrentCVS Platform: All OS/Version: All Status: NEW Severity: Critical Priority: Other Component: org.apache.xpath AssignedTo: [EMAIL PROTECTED] ReportedBy: [EMAIL PROTECTED] When using extensions, the method key that's generated for caching purposes is not unique. The current key is based on - current position in the m_opMap array + hash code + current time in milliseconds. However, the below debug statements show that they are not unique and as a result the invocation of methods are incorrect and the behavior is very non-deterministic. Steps to reproduce: ------------------ This is extremely hard to reproduce since the method key is dependent on the hash code and the current time. In order to reproduce these have to match up. Debug statements: ----------------- 1. Output of the method key from Compiler:compileExtension(). The key 624301046301758359 is duplicate. The value within parenthesis show the different components of the key i.e. current position in the m_opMap array, hash code and current time in milliseconds, each separated by a '-'. compileExtension-> methodKey=828971046301758312(8-2897-1046301758312) funcName=nodeset compileExtension-> methodKey=827741046301758328(8-2774-1046301758328) funcName=nodeset compileExtension-> methodKey=825811046301758343(8-2581-1046301758343) funcName=nodeset compileExtension-> methodKey=624881046301758359(6-2488-1046301758359) funcName=com.documentum.xml.xdql.DfXmlQuery.new compileExtension-> methodKey=624481046301758359(6-2448-1046301758359) funcName=init compileExtension-> methodKey=624301046301758359(6-2430-1046301758359) funcName=setDql compileExtension-> methodKey=626081046301758359(6-2608-1046301758359) funcName=includeContent compileExtension-> methodKey=625081046301758359(6-2508-1046301758359) funcName=includeContent compileExtension-> methodKey=624771046301758359(6-2477-1046301758359) funcName=setErrorTag compileExtension-> methodKey=624321046301758359(6-2432-1046301758359) funcName=setMaxRows compileExtension-> methodKey=624081046301758359(6-2408-1046301758359) funcName=setMetaDataAsAttributes compileExtension-> methodKey=623771046301758359(6-2377-1046301758359) funcName=setMetaDataAsAttributes compileExtension-> methodKey=623441046301758359(6-2344-1046301758359) funcName=setRowsetTag compileExtension-> methodKey=623321046301758359(6-2332-1046301758359) funcName=setContentEncoding compileExtension-> methodKey=623131046301758359(6-2313-1046301758359) funcName=setRepeatingAsNested compileExtension-> methodKey=625291046301758359(6-2529-1046301758359) funcName=setRepeatingAsNested compileExtension-> methodKey=624811046301758359(6-2481-1046301758359) funcName=execute compileExtension-> methodKey=624301046301758359(6-2430-1046301758359) funcName=getXMLDOM 2. The below debug statements show entry (>>) and exit (<<) calls. If the entry calls have any parameters they are shown within the parenthesis and if the exit calls have any return values they are shown using '-->'. It's clearly visible that because of the key duplication, the value in the cache is being replaced. Also, it's evident that an ArrayIndexOutOfBoundsException is thrown. However, this exception is swallowed in ExtensionHandlerJavaPackage and as a result the problem is lot harder to track. >> org.apache.xalan.extensions.ExtensionsTable:extFunction ('http://xml.apache.org/xslt/java', 'setDql', '[EMAIL PROTECTED]', '624301046301758359' , '[EMAIL PROTECTED]') >> org.apache.xalan.extensions.ExtensionHandlerJavaPackage:callFunction ('setDql', '[EMAIL PROTECTED]', '624301046301758359', 'XPathContext$XPathExpressionCont [EMAIL PROTECTED]') >> org.apache.xalan.extensions.ExtensionHandlerJavaPackage:getFromCache ('624301046301758359', '[EMAIL PROTECTED]', 'Object;@4ff') << org.apache.xalan.extensions.ExtensionHandlerJavaPackage:getFromCache --> null >> org.apache.xalan.extensions.MethodResolver:getMethod ('[EMAIL PROTECTED]', 'setDql', 'Object;@4ff', 'Object;@3a5', 'XPathContext$XPathExpress [EMAIL PROTECTED]', '2') >> org.apache.xalan.extensions.MethodResolver:scoreMatch ('Class;@33b', '0', 'Object;@4ff', '1000') << org.apache.xalan.extensions.MethodResolver:scoreMatch -- > "1000" >> org.apache.xalan.extensions.MethodResolver:convertParams ('Object;@4ff', 'Object;@3a5', 'Class;@33b', 'XPathContext$XPathExpressionContex [EMAIL PROTECTED]') >> org.apache.xalan.extensions.MethodResolver:convert ('[EMAIL PROTECTED]', '[EMAIL PROTECTED]') << org.apache.xalan.extensions.MethodResolver:convert -- > "select r_object_id, object_name, award_date, award_name, i_folder_id from bea_award where object_name = 'dave_award_test' or object_name = 'awd00002.xml' and folder('/BEAPublish/news_events', DESCEND)" << org.apache.xalan.extensions.MethodResolver:getMethod -- > "public void com.documentum.xml.xdql.DfXmlQuery.setDql(java.lang.String)" >> org.apache.xalan.extensions.ExtensionHandlerJavaPackage:putToCache ('624301046301758359', '[EMAIL PROTECTED]', 'Object;@4ff', '[EMAIL PROTECTED]') << org.apache.xalan.extensions.ExtensionHandlerJavaPackage:putToCache --> null >> com.documentum.xml.xdql.DfXmlQuery:setDql ('select r_object_id, object_name, award_date, award_name, i_folder_id from bea_award where object_name = 'dave_award_test' or object_name = 'awd00002.xml' and folder ('/BEAPublish/news_events', DESCEND)') << org.apache.xalan.extensions.ExtensionHandlerJavaPackage:callFunction --> null << org.apache.xalan.extensions.ExtensionsTable:extFunction --> null >> org.apache.xalan.extensions.ExtensionsTable:extFunction ('http://xml.apache.org/xslt/java', 'getXMLDOM', '[EMAIL PROTECTED]', '6243010463017583 59', '[EMAIL PROTECTED]') >> org.apache.xalan.extensions.ExtensionHandlerJavaPackage:callFunction ('getXMLDOM', '[EMAIL PROTECTED]', '624301046301758359', 'XPathContext$XPathExpressionC [EMAIL PROTECTED]') >> org.apache.xalan.extensions.ExtensionHandlerJavaPackage:getFromCache ('624301046301758359', '[EMAIL PROTECTED]', 'Object;@bb2') << org.apache.xalan.extensions.ExtensionHandlerJavaPackage:getFromCache -- > "public void com.documentum.xml.xdql.DfXmlQuery.setDql(java.lang.String)" >> org.apache.xalan.extensions.MethodResolver:convertParams ('Object;@bb2', 'Object;@ba1', 'Class;@b9f', 'XPathContext$XPathExpressionContex [EMAIL PROTECTED]') << org.apache.xalan.extensions.MethodResolver:convertParams --> with Exception: java.lang.ArrayIndexOutOfBoundsException at org/apache/xalan/extensions/MethodResolver.convertParams at org/apache/xalan/extensions/ExtensionHandlerJavaPackage.callFunction at org/apache/xalan/extensions/ExtensionsTable.extFunction at org/apache/xpath/functions/FuncExtFunction.execute at org/apache/xpath/XPath.execute at org/apache/xalan/templates/ElemVariable.getValue at org/apache/xalan/templates/ElemVariable.execute at org/apache/xalan/transformer/TransformerImpl.executeChildTemplates at org/apache/xalan/templates/ElemTemplate.execute at org/apache/xalan/templates/ElemCallTemplate.execute at org/apache/xalan/transformer/TransformerImpl.executeChildTemplates at org/apache/xalan/transformer/TransformerImpl.transformToRTF at org/apache/xalan/templates/ElemVariable.getValue at org/apache/xalan/templates/ElemVariable.execute at org/apache/xalan/transformer/TransformerImpl.executeChildTemplates at org/apache/xalan/transformer/TransformerImpl.applyTemplateToNode at org/apache/xalan/transformer/TransformerImpl.transformNode at org/apache/xalan/transformer/TransformerImpl.transform at org/apache/xalan/transformer/TransformerImpl.transform at org/apache/xalan/transformer/TransformerImpl.transform at com/documentum/operations/DfApplyTransformation.applyStyleSheet at com/documentum/operations/DfApplyTransformation.applyStyleSheet at com/documentum/operations/DfApplyTransformation.applyTransformation at com/documentum/operations/DfApplyTransformation.execute at com/documentum/operations/DfOperationStep.execute at com/documentum/operations/DfOperation.execute at com/documentum/wcm/WcmUtil.createTransToFile at com/documentum/webclient/xmledit/server/PreviewMgr.setPreviewContent at com/documentum/webclient/xmledit/server/DocbaseServer.request >> org.apache.xalan.extensions.MethodResolver:getMethod ('[EMAIL PROTECTED]', 'getXMLDOM', 'Object;@bb2', 'Object;@ba1', 'XPathContext$XPathExpr [EMAIL PROTECTED]', '2') >> org.apache.xalan.extensions.MethodResolver:scoreMatch ('Class;@389', '0', 'Object;@bb2', '1000') << org.apache.xalan.extensions.MethodResolver:scoreMatch -- > "1000" >> org.apache.xalan.extensions.MethodResolver:convertParams ('Object;@bb2', 'Object;@ba1', 'Class;@389', 'XPathContext$XPathExpressionContex [EMAIL PROTECTED]') << org.apache.xalan.extensions.MethodResolver:getMethod -- > "public org.w3c.dom.Document com.documentum.xml.xdql.DfXmlQuery.getXMLDOM()" >> org.apache.xalan.extensions.ExtensionHandlerJavaPackage:putToCache ('624301046301758359', '[EMAIL PROTECTED]', 'Object;@bb2', '[EMAIL PROTECTED]') #### WARNING: methodKey 624301046301758359 already exists and is being replaced << org.apache.xalan.extensions.ExtensionHandlerJavaPackage:putToCache --> "public void com.documentum.xml.xdql.DfXmlQuery.setDql(java.lang.String)" >> com.documentum.xml.xdql.DfXmlQuery:getXMLDOM << com.documentum.xml.xdql.DfXmlQuery:getXMLDOM --> "[#document: null]" << org.apache.xalan.extensions.ExtensionHandlerJavaPackage:callFunction -- > "[#document: null]" << org.apache.xalan.extensions.ExtensionsTable:extFunction -- > "[#document: null]" Proposed Solution: ----------------- 1. Generate a method key that's guaranteed to be unique. A solution is to use the hash code along with a sequential long value. Below is what the code would look like: private Expression compileExtension(int opPos) throws TransformerException { .... // We create a method key to uniquely identify this function so that we // can cache the object needed to invoke it. This way, we only pay the // reflection overhead on the first call. Function extension = new FuncExtFunction(ns, funcName, + String.valueOf(hashCode()) + String.valueOf(getNextMethodId())); .... } synchronized private long getNextMethodId() { if (s_nextMethodId == Long.MAX_VALUE) s_nextMethodId = Long.MIN_VALUE; return s_nextMethodId++; } private static long s_nextMethodId = Long.MIN_VALUE; 2. The code that calls into MethodResolver:convertParams() should not swallow exceptions blindly. Rather they should swallow only known exceptions such as InvocationTargetException. This pattern of code is evident in ExtensionHandlerJavaClass:callFunction() and ExtensionHandlerJavaPackage:callFunction().
