Hi I see what you mean but still there must be a leak somewhere…
The payload is small <RefData> <FinInstrmGnlAttrbts> <Id>DE000C3JALS0</Id> <FullNm>EAA3 SI 20211220 CS</FullNm> <ShrtNm>EEX/EUAA P AUCTION SPOT</ShrtNm> <ClssfctnTp>ITNXXX</ClssfctnTp> <NtnlCcy>EUR</NtnlCcy> <CmmdtyDerivInd>false</CmmdtyDerivInd> </FinInstrmGnlAttrbts> <Issr>529900J0JGLSFDWNFC20</Issr> <TradgVnRltdAttrbts> <Id>XEER</Id> <IssrReq>false</IssrReq> <FrstTradDt>2015-03-25T06:00:00Z</FrstTradDt> <TermntnDt>2031-12-30T23:59:59Z</TermntnDt> </TradgVnRltdAttrbts> <DerivInstrmAttrbts> <PricMltplr>500</PricMltplr> <AsstClssSpcfcAttrbts> <Cmmdty> <Pdct> <Envttl> <Emssns> <BasePdct>ENVR</BasePdct> <SubPdct>EMIS</SubPdct> <AddtlSubPdct>EUAA</AddtlSubPdct> </Emssns> </Envttl> </Pdct> <TxTp>OTHR</TxTp> <FnlPricTp>EXOF</FnlPricTp> </Cmmdty> </AsstClssSpcfcAttrbts> </DerivInstrmAttrbts> <TechAttrbts> <RlvntCmptntAuthrty>DE</RlvntCmptntAuthrty> <PblctnPrd> <FrDt>2021-03-18</FrDt> </PblctnPrd> <RlvntTradgVn>XEER</RlvntTradgVn> </TechAttrbts> </RefData> Is there a way to control the behaviour of @Xpath() as with .xpath() ? /M ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Tuesday, November 9th, 2021 at 06:40, Siano, Stephan <stephan.si...@sap.com.INVALID> wrote: > Hi > > I don’t think that this issue is related to thread safety. XPath as such is a > very expensive operation as it requires parsing the document into a DOM. You > have 10 of those XPath parameters and the heap dump shows 10 XPath builders > that are consuming a lot of memory. You’d probably better pass the payload > only once (maybe as a Node or Document) and then execute the XPath > expressions on it inside the bean (then you will only parse your document > once and have only one DOM tree). > > Best regards > > Stephan > > From: Mikael Andersson Wigander <mikael.andersson.wigan...@pm.me.INVALID> > Sent: Monday, 8 November 2021 20:30 > To: Camel Mail List <users@camel.apache.org> > Subject: Sv: How to make a bean thread safe? > > There’s a typo in the code sample. > > The processing SHOULD be parallel, not sequential as in the snippet. > > /M > > På mån, nov. 8, 2021 vid 20:11, Mikael Andersson Wigander > <mikael.andersson.wigan...@pm.me.INVALID> skrev: > >> Hi >> >> With the risk of being seen as a n00b (again)… >> >> We are processing large XML files (0.5GB/~500.000 records). >> >> To process them we use stream caching, spit, parallel processing, xpath and >> a bean. >> >> We get a lot of OutOfMemoryExceptions and after analysing we see that the >> call to the bean method is the villain. >> >> The process is to split() using tokenizeXML() on a tag that makes up one >> record in the XML. >> >> For each of these records we call a bean where the method utilises @Xpath() >> on the method parameters. >> >> We see in the heap dump that these calls are never GC'd, we have 90% >> leftovers >> >> The question is: is this related to a not thread safe bean/method or what >> could be the reason? >> >> The documentation states the default behaviour is a Signleton and when used >> in concurrent processing it must be thread safe… >> >> https://camel.apache.org/components/3.11.x/bean-component.html#_options >> >> Running as a war under Tomcat 9 on Windows using Camel 3.11.3 and Spring >> Boot 2.5.6. >> >> Server has 32GB of RAM… >> >> Route: >> >> from(file("Full")) >> >> .streamCaching() >> >> .unmarshal() >> >> .zipFile() >> >> .split() >> >> .tokenizeXML("RefData") >> >> .streaming() >> >> .parallelProcessing(false) >> >> .bean(XmlToSqlBean.class) >> >> .to(jdbc("default")) >> >> .end(); >> >> Bean: >> >> public class XmlToSqlBean { >> >> public String toSql(@XPath("//FinInstrmGnlAttrbts/Id") final String isin, >> >> @XPath("//NtnlCcy") final String currency, >> >> @XPath("//FullNm") final String fullName, >> >> @XPath("//TradgVnRltdAttrbts/Id") final String venue, >> >> @XPath("//ClssfctnTp") final String classification, >> >> @XPath("//TradgVnRltdAttrbts/TermntnDt") final String terminationDate, >> >> @XPath("//Issr") final String issuer, >> >> @XPath("//MtrtyDt") String maturityDate, >> >> @XPath("//TermntdRcrd") final String termnRecord, >> >> @XPath("//NewRcrd") final String newRecord) { >> >> … >> >> } >> >> } >> >> Thanks >> >> /M
