hansva opened a new issue, #9342: URL: https://github.com/apache/hop/issues/9342
### Apache Hop version? 2.20.0 ### Java version? 21 ### Operating system Other ### What happened? *XML join* and *XSD validator* (the transforms) repeat their XML setup for every row. Creating an XML factory is expensive in Hop: `DocumentBuilderFactory`, `SchemaFactory` and `XPathFactory.newInstance()` fall through to a ServiceLoader scan of every jar in `lib/core` (639 jars in the client), and `XmlParserFactoryProducer.createSecureDocBuilderFactory()` does that about 6 times per call, once for each security feature it sets. **XML join** (`XmlJoin.java`), for every row: - `XPathFactory.newInstance().newXPath()` at the top of `processRow()` (line 70) - `XmlParserFactoryProducer.createSecureDocBuilderFactory().newDocumentBuilder()` for every source row (lines 238-239), only to parse that row's XML fragment Profiling a 50,000-row join shows 91% of the CPU in creating these factories; parsing and joining is under 10%. The transform runs at about 3-4k rows/s. **XSD validator** (`XsdValidator.java`), for every row: - a new `SchemaFactory`, secure or not depending on *Allow external entities* (lines 139-143) - the XSD file is resolved, read and compiled again with `newSchema(...)` (lines 149-152 and 265-274), also when the XSD is a fixed file name - a new `Validator`, configured with the XXE features and entity resolver (line 155 and on) Profiling a 25,000-row validation shows 36% of the CPU looking up the schema factory, 37% compiling the XSD and 11% validating. The thread is on the CPU only about half the time, because it reads the XSD file for every row. The transform runs at about 1,000 rows/s. ### Issue Priority Priority: 2 ### Issue Component Component: Transforms -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
