RE: Warning "AJP13 protocol: Reuse is set to false" written logs every second of every day. Please help.
Hi Chris, A follow-up. I found the following discussion, which goes back to October 19, 2011: http://tomcat.10.x6.nabble.com/Reuse-mod-ajp-proxy-connections-td2096715.html I think it is relevant to this issue. Back then, one Dimitar said: >> "My understanding of the *reuse* flag in AJP13_END_RESPONSE >> structure is that this flag is hard coded and cannot be controlled >> by configuration settings. I couldn't find any in Tomcat AJP >> connector configuration that could do that." You responded with: >> "I'm no expert, but seeing keepalives disabled for a proxy connection >> would sure make me think that maybe that setting is disabling, ya >> know, the keepalive-style connection that AJP is supposed to maintain. >>... I'd try NOT disabling keepalives and see if that helps." In our set-up, the settings for socket_keepalive is false by default. So, we're now going to add the line worker.workerName.socket_keepalive=true to the workers.properties file and see what happens. Kind regards, Alfred -Oorspronkelijk bericht- Van: Alfred Bakia Verzonden: 17 June 2020 19:08 Aan: Tomcat Users List Onderwerp: RE: Warning "AJP13 protocol: Reuse is set to false" written logs every second of every day. Please help. Hi Chris, Thanks for your reply. Thanks also for your warning against interfering with the setting allowedRequestAttributesPattern ('Setting the value to ".*" is a violation of sane security policy'). I guessed as much, and am grateful for your confirmation. On the subject of mod_jk, we are apparently talking about the same thing. In our set-up the mod_jk log is called isapi_redirect.log. The code that generates the error is indeed /native/common/jk_ajp_common.c. The part where the error is generated is: case JK_AJP13_END_RESPONSE: ae->reuse = (int)jk_b_get_byte(msg); if (!ae->reuse) { /* * AJP13 protocol reuse flag set to false. * Tomcat will close its side of the connection. */ jk_log(l, JK_LOG_WARNING, "(%s) AJP13 protocol: Reuse is set to false", ae->worker->name); } This code snippet says that reuse is determined - at the end of a response - from a byte obtained from some message. Which raises two questions: 1) The response to the "ping" REST request has status code 204 ("No Content"). If, as you say, a 204 response is OK, then where would the reuse byte come from? 2) Do you know a way to set the "AJP13 protocol reuse flag" to true? As far as we know, the only "reuse" settings at our disposal are the current worker settings, worker.workerName.connection_pool_size=500 and worker. workerName.max_reuse_connections=250. Kind regards, Alfred -Oorspronkelijk bericht- Van: Christopher Schultz Verzonden: 16 June 2020 19:55 Aan: Tomcat Users List Onderwerp: Re: Warning "AJP13 protocol: Reuse is set to false" written logs every second of every day. Please help. -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Alfred, On 6/15/20 09:45, Alfred Bakia wrote: > Thanks, Chris. To respond to your remarks: > > 1) The warnings, "AJP13 protocol: Reuse is set to false", are being logged by Tomcat (in the Tomcat AJP connector logs). Weird. I can't find that string anywhere in code or other files in Tomcat, but its right there in mod_jk: ./native/common/jk_ajp_common.c:jk_log(l, JK_LOG_WARNING, "(%s) AJP13 protocol: Reuse is set to false", Are you sure you are looking at the right log file? > 2) As I mentioned earlier, the versions and settings of the servers > (and their respective IIS web servers) are the same. > > In any case I have discovered "how" the warning occurs. It is indeed > triggered by the REST API. > > The culprit is a REST request in the form: > > http://www.ourdomain.com/api/srv2/exercises/93/1431/26346/ping > > It is a POST request and "ping" is a custom REST method. It is just a > ping, so the status code of the response is 204 ("No Content"). A 204 response should not cause any problem. > I can confirm that every ping request results in the warning > "AJP13 protocol: Reuse is set to false" being written to Tomcat's > connector logs. > > Researching this on the web, I found the suggestion to add > allowedRequestAttributesPattern=".*" to the AJP connector in > server.xml. > > Is this a viable solution? This has nothing to do with your REST API. This is a change made to recent versions of Tomcat that may require you to allow certain non-standard variables to be passed-over from your web server to Tomcat via AJP. If your "ping" REST API is requiring some information to be passed from the web server to Tomcat and you are seeing errors o
RE: Warning "AJP13 protocol: Reuse is set to false" written logs every second of every day. Please help.
Hi Chris, Thanks for your reply. Thanks also for your warning against interfering with the setting allowedRequestAttributesPattern ('Setting the value to ".*" is a violation of sane security policy'). I guessed as much, and am grateful for your confirmation. On the subject of mod_jk, we are apparently talking about the same thing. In our set-up the mod_jk log is called isapi_redirect.log. The code that generates the error is indeed /native/common/jk_ajp_common.c. The part where the error is generated is: case JK_AJP13_END_RESPONSE: ae->reuse = (int)jk_b_get_byte(msg); if (!ae->reuse) { /* * AJP13 protocol reuse flag set to false. * Tomcat will close its side of the connection. */ jk_log(l, JK_LOG_WARNING, "(%s) AJP13 protocol: Reuse is set to false", ae->worker->name); } This code snippet says that reuse is determined - at the end of a response - from a byte obtained from some message. Which raises two questions: 1) The response to the "ping" REST request has status code 204 ("No Content"). If, as you say, a 204 response is OK, then where would the reuse byte come from? 2) Do you know a way to set the "AJP13 protocol reuse flag" to true? As far as we know, the only "reuse" settings at our disposal are the current worker settings, worker.workerName.connection_pool_size=500 and worker. workerName.max_reuse_connections=250. Kind regards, Alfred -Oorspronkelijk bericht- Van: Christopher Schultz Verzonden: 16 June 2020 19:55 Aan: Tomcat Users List Onderwerp: Re: Warning "AJP13 protocol: Reuse is set to false" written logs every second of every day. Please help. -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Alfred, On 6/15/20 09:45, Alfred Bakia wrote: > Thanks, Chris. To respond to your remarks: > > 1) The warnings, "AJP13 protocol: Reuse is set to false", are being logged by Tomcat (in the Tomcat AJP connector logs). Weird. I can't find that string anywhere in code or other files in Tomcat, but its right there in mod_jk: ./native/common/jk_ajp_common.c:jk_log(l, JK_LOG_WARNING, "(%s) AJP13 protocol: Reuse is set to false", Are you sure you are looking at the right log file? > 2) As I mentioned earlier, the versions and settings of the servers > (and their respective IIS web servers) are the same. > > In any case I have discovered "how" the warning occurs. It is indeed > triggered by the REST API. > > The culprit is a REST request in the form: > > http://www.ourdomain.com/api/srv2/exercises/93/1431/26346/ping > > It is a POST request and "ping" is a custom REST method. It is just a > ping, so the status code of the response is 204 ("No Content"). A 204 response should not cause any problem. > I can confirm that every ping request results in the warning > "AJP13 protocol: Reuse is set to false" being written to Tomcat's > connector logs. > > Researching this on the web, I found the suggestion to add > allowedRequestAttributesPattern=".*" to the AJP connector in > server.xml. > > Is this a viable solution? This has nothing to do with your REST API. This is a change made to recent versions of Tomcat that may require you to allow certain non-standard variables to be passed-over from your web server to Tomcat via AJP. If your "ping" REST API is requiring some information to be passed from the web server to Tomcat and you are seeing errors on the Tomcat side, then you may have to fiddle with the allowedRequestAttributesPattern. Setting the value to ".*" is a violation of sane security policy. But I see no evidence that your "Reuse is set to false" error is related (yet) to the allowedRequestAttributesPattern. If you see errors in the Tomcat log, please post them and we'll see. The allowedRequestAttributesPattern wasn't added to Tomcat until 9.0.31 so if you really are running 9.0.21, then adding allowedRequestAttributesPattern will accomplish nothing. - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl7pB3wACgkQHPApP6U8 pFg97w//T8XVtWlic/x/nUJHZUXfDs99Ou8cKcSFFI/AbBldd5k8vSxCTKg40wON 3bUooWG5GWZhVRNHH2JH0eyiKqhLmDtAzdnSp7xAkAW2fAZ2Zlv0VLb0AzJCTqRg e1Nd9R9Ii9mjcSU5+M2WrNSourUhOT0FVYaqpvlaN89XyetHSuVIUladDuwM8kFN 3ngAtkgmIYfxXcqIPXjoNZ+s8cr1MB0qk+KkiXyOCb8XgfmZUkBMRncgqMAgEff5 p6Z/1jGVv0+S7E0+HV1yqJpakiGjVswfIjbc2s89YnVL6bvyBqUnJl4HrmOHY0bV d3O/NQ3+vZ/Kma4e84TI5QKQx0KvQj0oBH41fFl0WmPjraKjGMTTfMCy9BjvLwdf hbTEbZaBRvn7Tr+iR5ksrvaJTxZD1ABMb7o0uksCsPQO8h3tl3s7L5O4g0P3+7kV /SiqDD+WyqkhmJuX86Y3MtSeMUTsg9RiXOZLLGF59TOZFeso/2O+OWYU/uImXw2X opWW38Vowhn8O9a94RbRA67EvJFiLdWwTDoLlnVP0ZxGkdOIow0EQWfnCDKaBXOd l+BdTG7zPbU8I3bw00cXGytyCYENt9uIZJ/XVVk
RE: Warning "AJP13 protocol: Reuse is set to false" written logs every second of every day. Please help.
Thanks, Chris. To respond to your remarks: 1) The warnings, "AJP13 protocol: Reuse is set to false", are being logged by Tomcat (in the Tomcat AJP connector logs). 2) As I mentioned earlier, the versions and settings of the servers (and their respective IIS web servers) are the same. In any case I have discovered "how" the warning occurs. It is indeed triggered by the REST API. The culprit is a REST request in the form: http://www.ourdomain.com/api/srv2/exercises/93/1431/26346/ping It is a POST request and "ping" is a custom REST method. It is just a ping, so the status code of the response is 204 ("No Content"). I can confirm that every ping request results in the warning "AJP13 protocol: Reuse is set to false" being written to Tomcat's connector logs. Researching this on the web, I found the suggestion to add allowedRequestAttributesPattern=".*" to the AJP connector in server.xml. Is this a viable solution? Alfred -Oorspronkelijk bericht- Van: Christopher Schultz Verzonden: 12 June 2020 19:49 Aan: Tomcat Users List Onderwerp: Re: Warning "AJP13 protocol: Reuse is set to false" written logs every second of every day. Please help. -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Alfred, On 6/12/20 05:52, Alfred Bakia wrote: > Thanks, Jon. I did in fact compare all the settings between the > servers, including the logging settings. They are exactly the same.> > But there is new information. In a detailed comparison of the Java > code between the servers, I spotted one difference. > Something sets apart the instance that is logging the Warning > "AJP13 protocol: Reuse is set to false". The instance includes a REST > API. As Sherlock Holmes said, "When you have eliminated all which is > impossible, then whatever remains, however improbable, must be the > truth." > > I am now looking into how the combination REST - Tomcat - IIS can > trigger the warning. ...but this is being logged on the IIS side, not the Tomcat side. It's very unlikely that the application is causing these log messages to be displayed. Same version(s) of IIS? Same versions of mod_jk? - -chris > -Oorspronkelijk bericht- Van: > jonmcalexan...@wellsfargo.com.INVALID > Verzonden: 11 June 2020 23:57 Aan: users@tomcat.apache.org > Onderwerp: RE: Warning "AJP13 protocol: Reuse is set to false" > written logs every second of every day. Please help. > > Perhaps also compare your logging sensitivity between the servers. > > > Dream * Excel * Explore * Inspire Jon McAlexander Asst Vice President > > Middleware Product Engineering Enterprise CIO | Platform Services > | Middleware | Infrastructure Solutions > > 8080 Cobblestone Rd | Urbandale, IA 50322 MAC: F4469-010 Tel > 515-988-2508 | Cell 515-988-2508 > > jonmcalexan...@wellsfargo.com > > > This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. > > -Original Message- From: Christopher Schultz > Sent: Thursday, June 11, 2020 12:55 PM > To: users@tomcat.apache.org Subject: Re: Warning "AJP13 > protocol: Reuse is set to false" written logs every second of every day. Please help. > > Alfred, > > On 6/11/20 11:57, Alfred Bakia wrote: >> Hi Everyone, > >> This is my very first mail to the users list since joining yesterday. >> Not an auspicious start. But I hope I will be able to contribute in >> future. I seek your help for a persistent issue in one of our >> ColdFusion instances. > >> Description of issue: ColdFusion 2018 is an application server that >> uses Tomcat 9.0.21. Our ColdFusion installation consists of >> instances. The instances are independent application servers, each >> with its own Tomcat installation and Java Virtual Machine. >> The Java version is 11.0.7. > >> Each ColdFusion instance serves web content via the web server IIS. >> We have configured an AJP connector for the communication between >> Tomcat and IIS. The relevant settings are > > >> * In server.xml > >> > redirectPort="8445" protocol="AJP/1.3" >> tomcatAuthentication="false" maxThreads="500" >> packetSize="65535"/> > > >> * In isapi_redirect.properties > >> iis_buffer_enable= true > > >> * In workers.properties > >> worker.list=sr1studierdr1
RE: Warning "AJP13 protocol: Reuse is set to false" written logs every second of every day. Please help.
Thanks, Jon. I did in fact compare all the settings between the servers, including the logging settings. They are exactly the same. But there is new information. In a detailed comparison of the Java code between the servers, I spotted one difference. Something sets apart the instance that is logging the Warning "AJP13 protocol: Reuse is set to false". The instance includes a REST API. As Sherlock Holmes said, "When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth." I am now looking into how the combination REST - Tomcat - IIS can trigger the warning. Regards, Alfred -Oorspronkelijk bericht- Van: jonmcalexan...@wellsfargo.com.INVALID Verzonden: 11 June 2020 23:57 Aan: users@tomcat.apache.org Onderwerp: RE: Warning "AJP13 protocol: Reuse is set to false" written logs every second of every day. Please help. Perhaps also compare your logging sensitivity between the servers. Dream * Excel * Explore * Inspire Jon McAlexander Asst Vice President Middleware Product Engineering Enterprise CIO | Platform Services | Middleware | Infrastructure Solutions 8080 Cobblestone Rd | Urbandale, IA 50322 MAC: F4469-010 Tel 515-988-2508 | Cell 515-988-2508 jonmcalexan...@wellsfargo.com This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. -Original Message- From: Christopher Schultz Sent: Thursday, June 11, 2020 12:55 PM To: users@tomcat.apache.org Subject: Re: Warning "AJP13 protocol: Reuse is set to false" written logs every second of every day. Please help. -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Alfred, On 6/11/20 11:57, Alfred Bakia wrote: > Hi Everyone, > > This is my very first mail to the users list since joining yesterday. > Not an auspicious start. But I hope I will be able to contribute in > future. I seek your help for a persistent issue in one of our > ColdFusion instances. > > Description of issue: ColdFusion 2018 is an application server that > uses Tomcat 9.0.21. Our ColdFusion installation consists of instances. > The instances are independent application servers, each with its own > Tomcat installation and Java Virtual Machine. The Java version is > 11.0.7. > > Each ColdFusion instance serves web content via the web server IIS. > We have configured an AJP connector for the communication between > Tomcat and IIS. The relevant settings are > > > * In server.xml > > redirectPort="8445" protocol="AJP/1.3" tomcatAuthentication="false" > maxThreads="500" packetSize="65535"/> > > > * In isapi_redirect.properties > > iis_buffer_enable= true > > > * In workers.properties > > worker.list=sr1studierdr1 > > worker.sr1studierdr1.type=ajp13 > > worker.sr1studierdr1.host=localhost > > worker.sr1studierdr1.port=8012 > > worker.sr1studierdr1.connection_pool_size=800 > > worker.sr1studierdr1.connection_pool_timeout=60 > > worker.sr1studierdr1.max_reuse_connections=400'' > > On one of the instances (name: 'sr1studierdr1'), the following WARNING > is written to isapi_redirect.log every second or so: > > > * [Thu Jun 11 16:44:57.739 2020] [11308:15392] [warn] > ajp_process_callback::jk_ajp_common.c (2242): (sr1studierdr1) AJP13 > protocol: Reuse is set to false > > Nevertheless, the application seems to work as intended. > > We're at a loss why this is happening only to this particular > instance. There are no such warnings in other instances that share > exactly the same settings. > > Do you know what is causing the warning, "AJP13 protocol: Reuse is set > to false", or how to solve this? I have no idea, but Google seemed able to come up with this: https://forums.iis.net/t/1229345.aspx?Error+AJP13+protocol+Reuse+is+set+ to+false+on+log+file+ Does that help at all? Apache httpd configuration of mod_jk has a DisableReuse flag that can be set, but I don't see such a thing for IIS. - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl7ib9wACgkQHPApP6U8 pFiTzxAAljUf3DhO5zCex044toYxhtsrjb0ARCpFggrOXWiKBHHAyvnsMe/d1CFb Tp5BLQdoWD1qorCkXMvFAYAdFOcbgmxMuUb3dkyHiq9JMZxINz3vOUXGtyqKfTLd IT8VZ+kUSUq3brcoqMdkCNNpILAVNprtwCJdMoPSilVufG2vksjbBS2PT6YzSsXS EaOb138vVb82HA6vvtOsi9EbOvh1cRVhZ2sIQlvrYsoTjeRD4QmRbmIQw+TcMPag gtZtf46TyAtQSOs1L50LxR
Warning "AJP13 protocol: Reuse is set to false" written logs every second of every day. Please help.
Hi Everyone, This is my very first mail to the users list since joining yesterday. Not an auspicious start. But I hope I will be able to contribute in future. I seek your help for a persistent issue in one of our ColdFusion instances. Description of issue: ColdFusion 2018 is an application server that uses Tomcat 9.0.21. Our ColdFusion installation consists of instances. The instances are independent application servers, each with its own Tomcat installation and Java Virtual Machine. The Java version is 11.0.7. Each ColdFusion instance serves web content via the web server IIS. We have configured an AJP connector for the communication between Tomcat and IIS. The relevant settings are * In server.xml * In isapi_redirect.properties iis_buffer_enable= true * In workers.properties worker.list=sr1studierdr1 worker.sr1studierdr1.type=ajp13 worker.sr1studierdr1.host=localhost worker.sr1studierdr1.port=8012 worker.sr1studierdr1.connection_pool_size=800 worker.sr1studierdr1.connection_pool_timeout=60 worker.sr1studierdr1.max_reuse_connections=400'' On one of the instances (name: 'sr1studierdr1'), the following WARNING is written to isapi_redirect.log every second or so: * [Thu Jun 11 16:44:57.739 2020] [11308:15392] [warn] ajp_process_callback::jk_ajp_common.c (2242): (sr1studierdr1) AJP13 protocol: Reuse is set to false Nevertheless, the application seems to work as intended. We're at a loss why this is happening only to this particular instance. There are no such warnings in other instances that share exactly the same settings. Do you know what is causing the warning, "AJP13 protocol: Reuse is set to false", or how to solve this? Kind regards, Alfred