This is the warning message we get when cachingAllowed is not set to false
org.apache.catalina.webresources.Cache.getResource Unable to add the resource
at [/WEB-INF/classes/] to the cache for web application [/xxxxx] because there
was insufficient free space available after evicting expired cache entries -
consider increasing the maximum size of the cache.
Regards
Jalaj P Asher
-----Original Message-----
From: Jalaj Asher
Sent: Tuesday, July 16, 2024 1:30 PM
To: Tomcat Users List <users@tomcat.apache.org>
Subject: RE: Reg: tomcat CPU spikes
> space". Which was very quickly filling up our disk space as well as
> increasing disk IO causing latency concerns.
1. Also interesting. Can you post one of those messages here? Was there a stack
trace shown or just the warning?
It is just the warning. No stack trace. I will work on recreating this
since all our environments has this disabled.
2. Interesting. How much static content do you have? This seems like a good
use-case for a reverse-proxy to handle your static content for you.
We have not collated the complete size of it. But are reasons we cannot do that.
Also I was reviewing some older heap dumps and I could see that the jars are
getting cached in tomcat even with cachingAllowed=false.
Also this is not a consistent issue once it happens it takes sometime for the
stack to go away as well as post tomcat reboots the problem goes away with the
same settings and we do see that the wars are getting deployed during tomcat
startup as well.
Regards
Jalaj P Asher
-----Original Message-----
From: Christopher Schultz <ch...@christopherschultz.net>
Sent: Tuesday, July 16, 2024 10:05 AM
To: users@tomcat.apache.org
Subject: Re: Reg: tomcat CPU spikes
Attention! - This email has originated from an External Source outside of
eClinicalWorks. Always use caution when opening attachments, clicking links, or
when responding to this email. If you feel this is a phishing scam, please use
the Phish Alert Report button in Outlook.
Jalaj,
On 7/15/24 18:18, Jalaj Asher wrote:
> We ran into 2 issues
> 1. We needed to allocate significant amount of -XMX for heap space,
> if we allowed caching, since increasing memory by a few hundred MB as
> well was not enough.
Interesting. How much static content do you have? This seems like a good
use-case for a reverse-proxy to handle your static content for you.
> 2. Also with the setting being enabled, it generated logs stating
> "could not add a resource <resource name> as there wasn’t enough
> space". Which was very quickly filling up our disk space as well as
> increasing disk IO causing latency concerns.
Also interesting. Can you post one of those messages here? Was there a stack
trace shown or just the warning?
-chris
> -----Original Message-----
> From: Christopher Schultz <ch...@christopherschultz.net>
> Sent: Monday, July 15, 2024 4:19 PM
> To: users@tomcat.apache.org
> Subject: Re: Reg: tomcat CPU spikes
>
> Attention! - This email has originated from an External Source outside of
> eClinicalWorks. Always use caution when opening attachments, clicking links,
> or when responding to this email. If you feel this is a phishing scam, please
> use the Phish Alert Report button in Outlook.
>
>
> Jalaj,
>
> On 7/15/24 15:03, Jalaj Asher wrote:
>> Yeah I was wondering the same as this has been in place since a few
>> years now atleast 4 years since cachingAllowed had some changes in
>> tomcat 8 which was resulting in it caching all static content as well
>> as jsps and jars and our though process was if we have static content
>> being cached on the client end and jsps in the work folder each time
>> on access we don’t need the cache.
> Does the cache actively hurt you?
>
>> Is there a way to cache just the jars and not every thing else in memory ?
>
> I think the short answer is "no there is not a way to do this" but I may be
> wrong.
>
> The long answer might be "maybe, but you will have to play games with
> <PreResources> and <PostResources> and maybe some other things to get it
> working.
>
> I would save yourself some complexity and simply enable caching.
>
> -chris
>
>> -----Original Message-----
>> From: Christopher Schultz <ch...@christopherschultz.net>
>> Sent: Friday, July 12, 2024 4:02 PM
>> To: users@tomcat.apache.org
>> Subject: Re: Reg: tomcat CPU spikes
>>
>> Attention! - This email has originated from an External Source outside of
>> eClinicalWorks. Always use caution when opening attachments, clicking links,
>> or when responding to this email. If you feel this is a phishing scam,
>> please use the Phish Alert Report button in Outlook.
>>
>>
>> Jalaj,
>>
>> On 7/12/24 10:19, Jalaj Asher wrote:
>>> Thank you Chuck and John for the responses.
>>>
>>> Just a few points from the things you highlighted and wanted me to
>>> check 1. unpackwar is set to true. I checked and was informed that we need
>>> that to be true for a specific war file.
>>> 2. cachingAllowed=false. We keep it as false across the board.
>>
>> Well... that'll do it. In order to locate resources, Tomcat needs to sift
>> through all of those JAR files every time. Scanning ZIP files is expensive.
>>
>> You might want to reconsider this particular setting in your environment.
>>
>>> Also the reason I shared 2 different stacks is to highlight that the
>>> problem does not occur post restart or with any specific part of the
>>> application like the parser going in a loop but it kicks of at
>>> random times impacting multiple tomcats. But all having the same
>>> stack waiting on archiveresourceset and java.util.zip.
>> My only question would be "why is this only now coming to your attention?
>> Things should have been behaving this way .. for a long time.
>>
>>> I am working on getting both types of stacks to share here as well.
>>>
>>> Had a question we just have one war file if unpack war is triggered
>>> why should it impact loading jars from the entire webapp lib ?
>>
>> Because WEB-INF/lib is full of JAR files that need to be scanned every tie
>> you try to load ... anything. Since you have disabled caching, it has to
>> re-check every file for every resource-load request.
>>
>> -chris
>>
>>> -----Original Message-----
>>> From: Chuck Caldarale <n82...@gmail.com>
>>> Sent: Wednesday, July 10, 2024 6:19 PM
>>> To: Tomcat Users List <users@tomcat.apache.org>
>>> Subject: Re: Reg: tomcat CPU spikes
>>>
>>> Attention! - This email has originated from an External Source outside of
>>> eClinicalWorks. Always use caution when opening attachments, clicking
>>> links, or when responding to this email. If you feel this is a phishing
>>> scam, please use the Phish Alert Report button in Outlook.
>>>
>>>
>>>> On Jul 10, 2024, at 17:02, Jalaj Asher
>>>> <jalaj.as...@eclinicalworks.com.INVALID> wrote:
>>>>
>>>> Sharing another stack to see if this can give any more insights.this
>>>> thread is the tomcat main thread was loading about 65MB of data.
>>>>
>>>> "main" #1 prio=5 os_prio=0
>>>> java.lang.Thread.State: RUNNABLE
>>>> at java.util.zip.ZipFile.getEntry(Native Method)
>>>> at java.util.zip.ZipFile.getEntry(ZipFile.java:328)
>>>> - locked <0x00000000a1b04418> (a java.util.jar.JarFile)
>>>> at java.util.jar.JarFile.getEntry(JarFile.java:253)
>>>> at java.util.jar.JarFile.getJarEntry(JarFile.java:236)
>>>> at
>>>> org.apache.catalina.webresources.AbstractSingleArchiveResourceSet.getArchiveEntry(AbstractSingleArchiveResourceSet.java:97)
>>>> at
>>>> org.apache.catalina.webresources.AbstractArchiveResourceSet.getResource(AbstractArchiveResourceSet.java:249)
>>>> at
>>>> org.apache.catalina.webresources.StandardRoot.getResourceInternal(StandardRoot.java:272)
>>>> at
>>>> org.apache.catalina.webresources.StandardRoot.getResource(StandardRoot.java:213)
>>>> at
>>>> org.apache.catalina.webresources.StandardRoot.getClassLoaderResource(StandardRoot.java:220)
>>>> at
>>>> org.apache.catalina.loader.WebappClassLoaderBase.findClassInternal(WebappClassLoaderBase.java:2348)
>>>> at
>>>> org.apache.catalina.loader.WebappClassLoaderBase.findClass(WebappClassLoaderBase.java:875)
>>>> at
>>>> org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1376)
>>>> - locked <0x00000000f24fc728> (a java.lang.Object)
>>>> at
>>>> org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1220)
>>>> at java.lang.Class.forName0(Native Method)
>>>> at java.lang.Class.forName(Class.java:348)
>>>> at
>>>> com.sun.beans.finder.ClassFinder.findClass(ClassFinder.java:67)
>>>> at
>>>> com.sun.beans.finder.ClassFinder.findClass(ClassFinder.java:110)
>>>> at
>>>> com.sun.beans.finder.InstanceFinder.instantiate(InstanceFinder.java:93)
>>>> at
>>>> com.sun.beans.finder.InstanceFinder.find(InstanceFinder.java:66)
>>>> at
>>>> java.beans.Introspector.findExplicitBeanInfo(Introspector.java:448)
>>>> at java.beans.Introspector.<init>(Introspector.java:398)
>>>> at
>>>> java.beans.Introspector.getBeanInfo(Introspector.java:173)
>>>> at
>>>> org.springframework.beans.CachedIntrospectionResults.getBeanInfo(Ca
>>>> c
>>>> h
>>>> edIntrospectionResults.java:255)
>>>
>>>
>>> Is there some configuration setting in Spring that would disable
>>> caching? (I’m not really knowledgeable about Spring.)
>>>
>>> There should be more in the stack trace that would show what’s triggering
>>> the getBeanInfo() calls. Tomcat won’t be doing the lookup unless something
>>> asks for it.
>>>
>>> - Chuck
>>>
>>>
>>> --------------------------------------------------------------------
>>> - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>>
>>>
>>> CONFIDENTIALITY NOTICE TO RECIPIENT: This transmission contains
>>> confidential information belonging to the sender that is legally privileged
>>> and proprietary and may be subject to protection under the law, including
>>> the Health Insurance Portability and Accountability Act (HIPAA). If you are
>>> not the intended recipient of this e-mail, you are prohibited from sharing,
>>> copying, or otherwise using or disclosing its contents. If you have
>>> received this e-mail in error, please notify the sender immediately by
>>> reply e-mail and permanently delete this e-mail and any attachments without
>>> reading, forwarding or saving them. Thank you.
>>>
>>> CONFIDENTIALITY NOTICE TO RECIPIENT: This transmission contains
>>> confidential information belonging to the sender that is legally privileged
>>> and proprietary and may be subject to protection under the law, including
>>> the Health Insurance Portability and Accountability Act (HIPAA). If you are
>>> not the intended recipient of this e-mail, you are prohibited from sharing,
>>> copying, or otherwise using or disclosing its contents. If you have
>>> received this e-mail in error, please notify the sender immediately by
>>> reply e-mail and permanently delete this e-mail and any attachments without
>>> reading, forwarding or saving them. Thank you.
>>>
>>> --------------------------------------------------------------------
>>> - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
>>
>> CONFIDENTIALITY NOTICE TO RECIPIENT: This transmission contains confidential
>> information belonging to the sender that is legally privileged and
>> proprietary and may be subject to protection under the law, including the
>> Health Insurance Portability and Accountability Act (HIPAA). If you are not
>> the intended recipient of this e-mail, you are prohibited from sharing,
>> copying, or otherwise using or disclosing its contents. If you have received
>> this e-mail in error, please notify the sender immediately by reply e-mail
>> and permanently delete this e-mail and any attachments without reading,
>> forwarding or saving them. Thank you.
>>
>> CONFIDENTIALITY NOTICE TO RECIPIENT: This transmission contains confidential
>> information belonging to the sender that is legally privileged and
>> proprietary and may be subject to protection under the law, including the
>> Health Insurance Portability and Accountability Act (HIPAA). If you are not
>> the intended recipient of this e-mail, you are prohibited from sharing,
>> copying, or otherwise using or disclosing its contents. If you have received
>> this e-mail in error, please notify the sender immediately by reply e-mail
>> and permanently delete this e-mail and any attachments without reading,
>> forwarding or saving them. Thank you.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>
> CONFIDENTIALITY NOTICE TO RECIPIENT: This transmission contains confidential
> information belonging to the sender that is legally privileged and
> proprietary and may be subject to protection under the law, including the
> Health Insurance Portability and Accountability Act (HIPAA). If you are not
> the intended recipient of this e-mail, you are prohibited from sharing,
> copying, or otherwise using or disclosing its contents. If you have received
> this e-mail in error, please notify the sender immediately by reply e-mail
> and permanently delete this e-mail and any attachments without reading,
> forwarding or saving them. Thank you.
>
> CONFIDENTIALITY NOTICE TO RECIPIENT: This transmission contains confidential
> information belonging to the sender that is legally privileged and
> proprietary and may be subject to protection under the law, including the
> Health Insurance Portability and Accountability Act (HIPAA). If you are not
> the intended recipient of this e-mail, you are prohibited from sharing,
> copying, or otherwise using or disclosing its contents. If you have received
> this e-mail in error, please notify the sender immediately by reply e-mail
> and permanently delete this e-mail and any attachments without reading,
> forwarding or saving them. Thank you.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
CONFIDENTIALITY NOTICE TO RECIPIENT: This transmission contains confidential
information belonging to the sender that is legally privileged and proprietary
and may be subject to protection under the law, including the Health Insurance
Portability and Accountability Act (HIPAA). If you are not the intended
recipient of this e-mail, you are prohibited from sharing, copying, or
otherwise using or disclosing its contents. If you have received this e-mail in
error, please notify the sender immediately by reply e-mail and permanently
delete this e-mail and any attachments without reading, forwarding or saving
them. Thank you.
CONFIDENTIALITY NOTICE TO RECIPIENT: This transmission contains confidential
information belonging to the sender that is legally privileged and proprietary
and may be subject to protection under the law, including the Health Insurance
Portability and Accountability Act (HIPAA). If you are not the intended
recipient of this e-mail, you are prohibited from sharing, copying, or
otherwise using or disclosing its contents. If you have received this e-mail in
error, please notify the sender immediately by reply e-mail and permanently
delete this e-mail and any attachments without reading, forwarding or saving
them. Thank you.
