[OT] Re: DBAs?

[Chris Cheshire]

On Apr 2, 2023, at 4:41 PM, Kevin Huntly  wrote:Are there any DBAs in here? If so, are you aware of a MySQL user mailinglist? I'm having an issue with some stored procedures and need some helpHave you tried the forums?MySQL Forumsforums.mysql.com

Re: Conditional access logs

[Chris Cheshire]

> On Apr 22, 2022, at 1:32 PM, Tim Funk  wrote:
> 
> I don't think there is a technical reason why it couldn't be added. I think
> the hard part is getting the config wording/naming correct
> 
> Alternatively, I think an out of the box workaround could also be Tomcat's
> RewriteValve where the condition matches on header and sets the
> "environment variable"
> https://tomcat.apache.org/tomcat-9.0-doc/rewrite.html
> 
> You can submit a feature enhancement request in the bug database
> https://tomcat.apache.org/bugreport.html#How_to_submit_patches_and_enhancement_requests
> 
> -Tim


Good to know. I’m going to try the logback route first - if that fails I’ll 
look at an enhancement request or patch.

Chris


> 
> 
>> On Thu, Apr 21, 2022 at 5:08 PM Chris Cheshire  wrote:
>> 
>> Tomcat 9.
>> 
>> I wanted to separate out access logs for external api calls to log
>> different info than the standard access log line. For example, the api key
>> used which is set as a request header.
>> 
>> Adding that to the pattern was easy.
>> 
>> However the conditional logging was clunky. I found the ‘conditionIf’ and
>> ‘conditionUnless’ attributes for the access log valve, but these only work
>> on request attributes, not headers (at least that’s what the documentation
>> says).
>> 
>> I have created a filter that copies the values from the request headers to
>> equivalent  attributes, just so the condition can work. This is where it
>> feels  clunky, especially since the access log valve has replacement
>> parameters for logging request headers.
>> 
>> Is there a technical reason why the condition checking can’t work on
>> request headers in the valve? If not, can this be considered as a feature
>> request please?
>> 
>> 
>> 

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Conditional access logs

[Chris Cheshire]

> On Apr 22, 2022, at 7:21 PM, Aleks  wrote:
> 
> Hi.
> 
>> On Thu, 21 Apr 2022 17:07:27 -0400
>> Chris Cheshire  wrote:
>> 
>> Tomcat 9.
>> 
>> I wanted to separate out access logs for external api calls to log different
>> info than the standard access log line. For example, the api key used which
>> is set as a request header.
>> 
>> Adding that to the pattern was easy. 
>> 
>> However the conditional logging was clunky. I found the ‘conditionIf’ and
>> ‘conditionUnless’ attributes for the access log valve, but these only work on
>> request attributes, not headers (at least that’s what the documentation 
>> says).
>> 
>> I have created a filter that copies the values from the request headers to
>> equivalent  attributes, just so the condition can work. This is where it
>> feels  clunky, especially since the access log valve has replacement
>> parameters for logging request headers.
> 
> I have solved a similar issue with https://logback.qos.ch/access.html and
> https://logback.qos.ch/manual/loggingSeparation.html
> 
> That's the relevant snipplet for logback-access.xml. It uses the powerful
> filter feature from logback https://logback.qos.ch/manual/filters.html .
> 
> I attach the configmap definition which shows the whole config.
> 
> ```
>  
>
>
>  
>
>  if(event.getRequestURI().contains("/health"))
>return true;
>  return false;
>
>  
>  DENY
>
> ```
> 
> The log back access filter can be used to filter based on some AccessEvent 
> like
> a request header.
> 
> https://logback.qos.ch/manual/filters.html#access_EvalutorFilter
> https://logback.qos.ch/xref/ch/qos/logback/access/spi/AccessEvent.html
> 
> I think that you can also define based on the request header which logger
> should be used but I never had such a requirement .
> 
> Just replace the tomcat access valve with the logback one.
> 
> ```
> quiet="false" filename="conf/logback-access.xml"/>
> ```
> 
> It was required to add this lib to the ext dir of tomcat.
> Maybe there are some newer versions available.
> 
> ```
> ls -1 /opt/apache/tomcat/base/lib/ext/
> commons-compiler-3.1.3.jar
> jackson-annotations-2.9.9.jar
> jackson-core-2.9.9.jar
> jackson-databind-2.9.9.3.jar
> janino-3.1.3.jar
> logback-access-1.2.3.jar
> logback-core-1.2.3.jar
> logstash-logback-encoder-6.6.jar
> 
> ```
> 
>> Is there a technical reason why the condition checking can’t work on request
>> headers in the valve? If not, can this be considered as a feature request
>> please?
>> 
>> Thanks
> 
> 
> Regards
> Alex


Thanks, I’ll take a look at this. I already use logback for the app logging so 
this is familiar. I forgot they even had an access logger.

Chris
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Conditional access logs

[Chris Cheshire]

Tomcat 9.

I wanted to separate out access logs for external api calls to log different 
info than the standard access log line. For example, the api key used which is 
set as a request header.

Adding that to the pattern was easy. 

However the conditional logging was clunky. I found the ‘conditionIf’ and 
‘conditionUnless’ attributes for the access log valve, but these only work on 
request attributes, not headers (at least that’s what the documentation says).

I have created a filter that copies the values from the request headers to 
equivalent  attributes, just so the condition can work. This is where it feels  
clunky, especially since the access log valve has replacement parameters for 
logging request headers.

Is there a technical reason why the condition checking can’t work on request 
headers in the valve? If not, can this be considered as a feature request 
please?

Thanks
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Checksum Validation of Release 9.0.53 on a Mac

[Chris Cheshire]

> On Sep 23, 2021, at 10:48 AM, Darryl Philip Baker 
>  wrote:
> 
> I am trying to download and validate the binary release 53 file name is 
> apache-tomcat-9.0.53.tar.gz using openssl and the SHA256 checksum from the 
> link on the download web page. The target system for the installation is 
> running RHEL7. The checksums don’t match. I don’t know if I am doing things 
> wrong or there is an issue. If I am doing something wrong please let me know.
> 
> Here is what I am doing and the output on my Mac (11.6):
> $ /usr/bin/openssl sha256 apache-tomcat-9.0.53.tar.gz
> SHA256(apache-tomcat-9.0.53.tar.gz)= 
> 7b3e456ed76b0b42d99767985dc3774b22e2388834994f8539272eb7c05ab6fd
> $
> 
> The displayed checksum from the link on the download page is:
> ee731b2d0c3ab7e14fa924dcb8d598707cedf714c9ce1f2c2d907a1fd51e26f7eec1343c3dc39e240ff64dac2e0213f154d23e5f94a430f429165b5df51f786f
> 
Darryl,

The checksums are pgp and sha512.

On my Mac I used

$ shasum -a 512 apache-tomcat-9.0.53.tar.gz

and the checksums match.

Chris


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Reload rewrite rules

[Chris Cheshire]

Tomcat 9.0.45 - is there a way to reload the config for the rewrite valve at 
runtime without reloading the web app entirely? JMX operation perhaps?
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [OT] visualvm time stamps

[Chris Cheshire]

> On Apr 12, 2021, at 5:00 PM, Christopher Schultz 
>  wrote:
> 
> Chris,
> 
> On 4/12/21 12:50, Chris Cheshire wrote:
>>>> On Apr 9, 2021, at 3:02 PM, Chris Cheshire  wrote:
>>> My googlefu is failing me here.
>>> I am trying to figure out some anomalous database connection behavior in my 
>>> tomcat web app. I have enabled JMX/RMI and have visualvm running on my 
>>> local machine.
>>> I found the ability to monitor the active connections as a live chart, and 
>>> it has an export data function. This export creates a csv with what is 
>>> supposed to be a time stamp and a count but the time stamp is in a 5.6 
>>> format. I have never seen this before. How do I convert this into something 
>>> normal - millis since epoch or even a human readable ISO format?
>>> Example
>>> 44295.607552
>>> Chris
>> When all else fails ... RTFS
>> So it’s the number of days since 1900.  Why? Because Excel (and Lotus 1-2-3)
> 
> Well... they chose to use days-since-epoch and chose a different epoch than 
> you did. Since it's all arbitrary and relative, why is yours better than 
> theirs?

Because it doesn’t include a kludgy bug fix for a broken assumption about leap 
years and 1900 ;)

In any case, yes it’s just a different epoch. In 20+ years of writing software 
I have never come across it and was completely baffled at how to make sense of 
it and my googlefu failed me.




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [OT] visualvm time stamps

[Chris Cheshire]

> On Apr 9, 2021, at 3:02 PM, Chris Cheshire  wrote:
> 
> My googlefu is failing me here.
> 
> I am trying to figure out some anomalous database connection behavior in my 
> tomcat web app. I have enabled JMX/RMI and have visualvm running on my local 
> machine.
> 
> I found the ability to monitor the active connections as a live chart, and it 
> has an export data function. This export creates a csv with what is supposed 
> to be a time stamp and a count but the time stamp is in a 5.6 format. I have 
> never seen this before. How do I convert this into something normal - millis 
> since epoch or even a human readable ISO format? 
> 
> Example
> 44295.607552
> 
> Chris
> 

When all else fails ... RTFS

So it’s the number of days since 1900.  Why? Because Excel (and Lotus 1-2-3)

/facepalm

https://github.com/oracle/visualvm/blob/cbfb4426b25637fb5466705f76271cfcb5209090/plugins/mbeans/src/org/graalvm/visualvm/modules/mbeans/Formatter.java#L125

Re: [OT] visualvm time stamps

[Chris Cheshire]

> On Apr 11, 2021, at 12:24 PM, Suvendu Sekhar Mondal  wrote:
> 
> Hi Chris,
> 
>> On Sat, Apr 10, 2021 at 12:33 AM Chris Cheshire  wrote:
>> 
>> My googlefu is failing me here.
>> 
>> I am trying to figure out some anomalous database connection behavior in my 
>> tomcat web app. I have enabled JMX/RMI and have visualvm running on my local 
>> machine.
>> 
>> I found the ability to monitor the active connections as a live chart, and 
>> it has an export data function. This export creates a csv with what is 
>> supposed to be a time stamp and a count but the time stamp is in a 5.6 
>> format. I have never seen this before. How do I convert this into something 
>> normal - millis since epoch or even a human readable ISO format?
>> 
>> Example
>> 44295.607552
>> 
> 
> As far as I know, out of the box visualvm(I have 2.0.2) do not have
> any option to export CPU/GC/Heap details. Are you using any plugins to
> export data?
> 


Sorry - using the mbeans plugin. For this example, I am looking at the “active” 
attribute of a data source. Double click on the value to bring up a real time 
chart. Right click on the chart and select “save data as ...” to export as a 
CSV.



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

[OT] visualvm time stamps

[Chris Cheshire]

My googlefu is failing me here.

I am trying to figure out some anomalous database connection behavior in my 
tomcat web app. I have enabled JMX/RMI and have visualvm running on my local 
machine.

I found the ability to monitor the active connections as a live chart, and it 
has an export data function. This export creates a csv with what is supposed to 
be a time stamp and a count but the time stamp is in a 5.6 format. I have never 
seen this before. How do I convert this into something normal - millis since 
epoch or even a human readable ISO format? 

Example
44295.607552

Chris


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Join tables from different databases with tomcat datasource

[Chris Cheshire]

> On Jan 11, 2021, at 1:19 PM, Jerry Malcolm  wrote:
> 
> I have a query that needs to access tables in two different databases on the 
> same mysql instance.  I see how that can be done with the JOIN syntax in 
> mysql.  But datasource pools reference a single database, correct?  I really 
> need to 'join' two datasources for one sql call. I found info on doing this 
> by embedding TC in the Spring framework.  But I was hoping to not have to do 
> a major restructuring.  Is there a way to get a connection that attaches to 
> two separate databases?
> 
> 

If the databases are on the same MySQL instance, just adjust the grants for the 
user to be able to access tables in both datasources. 


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: File "catalina.out" not being created/populated when using Tomcat 9.0.31 + Ubuntu 20.04, and content goes to the Ubuntu syslog instead?

[Chris Cheshire]

> On Jun 22, 2020, at 4:44 AM, calder  wrote:
> 
> On Mon, Jun 22, 2020, 01:04 Brian  wrote
> 
> [ snip ]
> 
> - For some reason, the people at Ubuntu/Debian/Linux decided that Tomcat's
>> log should be found inside syslog, instead of staying independent inside
>> "catalina.out". Why is that? I don't know and I don't like it!
> 
> [ snip ]
> .
> Sorry - don't have a specific answer for your Ubuntu implementation.
> .
> However, this is one reason we do not use "distro-specific" Tomcat
> installations (to include implementations of WebSphere and WebLogic).
> .
> For example, we grab the plain vanilla Tomcat ZIP and extract it to "/opt/"
> (as in "/opt/tomcat/") - we now have complete control over its
> configuration and runtime instantiation.

+1

Also it is far better to use a logging framework than writing directly to 
System.out as you have more control over when and where the logs are written. 
You can use java.util.logging (juli) directly or even a facade like SLF4j and 
then use whatever implementation you like behind it. The advantage of the 
facade is that the implementation (logback, log4j, juli) can be changed with 
configuration and a new jar instead of code changes everywhere you write to the 
logs.

With either method you can write logs at different levels (trace, debug, info, 
warn error - juli does use different names though, same idea) and turn them 
on/off in configuration and even dynamically.

Chris
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: ajp connector, nio vs nio2

[Chris Cheshire]

On Fri, Mar 13, 2020 at 10:09 AM Mark Thomas  wrote:
>
> On 13/03/2020 13:54, Chris Cheshire wrote:
> > Using 9.0.31 on Java 8, I have my AJP connector configured as
> >
> >  > secretRequired="false" />
> >
> > According to the logs, this is defaulting to the NIO protocol.
>
> Correct.
>
> > The
> > connector comparison chart [1] implies that NIO2 is used for 8.5x
> > onwards.
>
> No, the Tomcat version line in that chart indicates from which version
> onwards the connector is available.
>
> > Shouldn't the AJP/1.3 protocol alias be using NIO2 by default (I don't
> > have APR/native installed)?
>
> No. It should use NIO.
>
> > What are the fundamental differences
> > between the two protocols?
>
> They work in fundamentally different ways (NIO - Poller vs NIO2 -
> callback) but for AJP, the actual difference in terms of performance,
> scalability etc is minimal. Personally, I'd stick with NIO.
>
> Mark
>
>

Thanks :)

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

ajp connector, nio vs nio2

[Chris Cheshire]

Using 9.0.31 on Java 8, I have my AJP connector configured as



According to the logs, this is defaulting to the NIO protocol. The
connector comparison chart [1] implies that NIO2 is used for 8.5x
onwards.
Shouldn't the AJP/1.3 protocol alias be using NIO2 by default (I don't
have APR/native installed)? What are the fundamental differences
between the two protocols?

Chris

PS Yes I have the AJP port only on localhost and firewalled off :)

[1] http://tomcat.apache.org/tomcat-9.0-doc/config/ajp.html#Connector_Comparison

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: CrawlerSessionManagerValve

[Chris Cheshire]

On Fri, Feb 28, 2020 at 12:51 PM Christopher Schultz
 wrote:
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris and Mark,
>
> On 2/28/20 11:51, Mark Thomas wrote:
> > On 28/02/2020 14:51, Chris Cheshire wrote:
> >> (9.0.31)
> >>
> >> What is the reason why the pattern isn't compiled with the case
> >> insensitive flag? Is it due to performance?
> >
> > I wrote that Valve. At least the first iteration anyway. Others
> > improved it along the way.
> >
> > I honestly can't remember why I opted for [bB]bot rather than
> > using CASE_INSENSITIVE.
> >
> > I do remember that the focus was on fixing an issue we (the ASF)
> > were having with our public Jira instance at the time in that bots
> > were generating huge numbers of sessions and, in turn, using up
> > large amounts of memory.
> >
> > Looking at it with the benefit of hindsight I'd worry about: -
> > performance - avoiding false positives
> >
> > There probably isn't much in it but I'd expect the current solution
> > is the right one for both of those. Unless you have a very
> > different UA pattern, in which case CASE_INSENSITIVE might help.
> > But I am guessing about the performance which really isn't the done
> > thing.
> >
> > If someone was to demonstrate that there was a measurable
> > performance benefit to some realistic patterns to using
> > CASE_INSENSITIVE then I'd support an enhancement to add an
> > attribute to specify the flags to use when compiling the pattern.
>

More of a curiosity. I am doing some crawler checking in my webapp. I
have an grossly repetitive regex and I was looking at this valve as an
example to optimize things a bit. I figured if it was a CI check then
it would negate the need for patterns like [bB].

There are a couple of common patterns that it is leaving out though :

.*[sS]p[iy]der.*
.*facebookexternalhit.*
.*(Mediapartners|Feedfetcher)-[gG]oogle.*

(last one is adding 'mediapartners' to the subpattern already in your
default regex)

> You can always use the (?i) flag-enabler if you want to use
> case-insensitive matches without changing the code.
>

+1 Did not know about this! If the flags can be specified in the
pattern itself, then there probably isn't much need for adding extra
attributes to the valve to achieve it.

Java regex tutorial[1] does say there is a slight performance hit for
a CI check, but it's not quantified. With processing speed increases,
my guess is it is completely negligible per request.

Chris

[1] https://docs.oracle.com/javase/tutorial/essential/regex/pattern.html

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

CrawlerSessionManagerValve

[Chris Cheshire]

(9.0.31)

What is the reason why the pattern isn't compiled with the case
insensitive flag? Is it due to performance?

Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: At wits end: Difficulties with IIS ISAPI connector and Tomcat

[Chris Cheshire]

On Mon, Feb 24, 2020 at 3:19 PM Ellen Meiselman  wrote:
>
> Hi,
>
> I’m having a lot of trouble configuring the isapi_redirect connector between 
> IIS and Tomcat. I am running out of ideas so it’s time to ask for help from 
> the experts. I think the problems remaining are in the tomcat configuration 
> area, not the IIS area anymore.
>
> What’s wrong:
> The ISAPI module appears to be working and correctly sending AJP requests to 
> Tomcat on port 8009, at which point Tomcat refuses those requests with a 403 
> error. The isapi_redirect.log shows the complete content of the tomcat 
> response, and no longer shows any errors - in other words, it thinks it is 
> working.
>
> Text of the 403 error:
>
>  HTTP Status 403 – Forbidden
>  Type Status Report
>  Description The server understood the request but refuses to authorize 
> it.
>  Apache Tomcat/8.5.51
>
>
> What does work:
> Requests directly to Tomcat on port 8080 to pages within the 
> connector-exposed web application work fine.
> For example, both of these work:
> localhost:8080/exposedApplication/simple.html. (viewed on the server’s 
> browser)
> my.servers.domain.com:8080/exposedApplication/simple.html (viewed anywhere 
> else)
>
>
> What does not work:
> Requests that go through IIS and the connector to the connector-exposed 
> application result in a 403 error.
> For example, this does not work:
> https:my.servers.domain.com/exposedApplication/simple.html
>
>
> This Windows 2019 setup has the following versions of tomcat, windows, etc:
>
> Tomcat version 8.5.51
> Isapi_redirect.dll version 1.2.46.0
> IIS 10/Windows server 2019
>
> I also have two older, similar Windows Server environments that work 
> perfectly. They both use these versions:
>
> Tomcat version 8.5.3 (64 bit) as a service
> Isapi_redirect.dll version 1.2.40.0 64 bit
> IIS 8/Windows server 2012R2
>
>
> The component versions between the working and non-working environments are 
> slightly different, and I think that might be the source of the problem - 
> there are probably new configuration requirements that I need to be aware of. 
> I started with the settings used in the working environments and found that 
> some things needed to be changed to get the connector to work at alll. For 
> example I had to specify an iPv4 address for the connector where I didn’t 
> need to before.
>
> My theories at the moment:
> 1. Maybe allowedRequestAttributesPattern is a problem? I saw a note about the 
> allowedRequestAttributesPattern attribute for the AJP connector possibly 
> causing a 403 error, but I don’t understand how to use it or if it is needed.
> 2. It’s possible that something in the Tomcat permissions settings are wrong, 
> but I really don’t know where to look.
>
>
> Relevant configuration settings in server.xml, workers.properties and 
> uriworkermap.properties:
>
> server.xml
>
>  redirectPort="8443" />
>  requiredSecret="true"  secret=“" redirectPort="8443" />
>
>   autoDeploy="true">
>  directory="logs"
>prefix="localhost_access_log" suffix=".txt"
>pattern="%h %l %u %t %r %s %b" />
>   
>
>   autoDeploy="true">
>  directory="logs"
> prefix="127_0_01_access_log" suffix=".txt"
> pattern="%h %l %u %t %r %s %b" />
>  
>
>
> workers.properties
>
> # Set properties for worker1 (ajp13)
> worker.worker1.type=ajp13
> worker.worker1.host=127.0.0.1
> worker.worker1.port=8009
> worker.worker1.secret=
>
>
> uriworkermap.properties
> /exposedApplication/*=worker1
>
>
> Any suggestions or new directions will be welcome.
>
> Thank you,
>
> Ellen Meiselman
>

Change requiredSecret="true" to secretRequired="true" in your AJP
connector definition.

HTH

Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: lower/uppercase rewrite maps

[Chris Cheshire]

On Thu, Jan 9, 2020 at 11:15 AM Felix Schumacher
 wrote:
>
>
> Am 09.01.20 um 17:01 schrieb Chris Cheshire:
> > Looking through the documentation for the rewrite valve [1], I see
> > there is an example of how to write and use a rewrite map to convert a
> > value to upper case. This is the inverse of what I want (lowercase),
> > so great, easy enough to implement. This seems like something that
> > could be included by default but I couldn't see anything in
> > catalina.jar.
> >
> > Is this something that would be included if I create a patch for it,
> > and how would I go about it?
>
> I have opened a PR a bit ago (https://github.com/apache/tomcat/pull/221)
> but hadn't had time to investigate any further. Remy thought it would be
> a bit overengineered. Romain liked the idea of ServiceLoader but wanted
> to have it a bit more optimized (see
> https://lists.apache.org/thread.html/472e875a46e811370f7df8b7d4fae37170a31d73c3d814a48e4d565c%40%3Cdev.tomcat.apache.org%3E).
>
> Would this be something you like to have?
>
> I think of committing the first part of the PR in any case, as I believe
> that the parsing of the parameters should be more in line with that of
> httpd.
>
> Felix
>

>From the example in the docs
**
RewriteMap uc example.maps.UpperCaseMap
RewriteRule ^/(.*)$ ${uc:$1}
**

All I would like is for UpperCaseMap to live within the Catalina
packaging as written so that I don't have to introduce a compile time
dependency for a configuration file entry. Maybe I am missing
something, but I don't see where having an SPI for this gains anything
for simple usages like this.

I can solve my problem by using multiple regular expressions, so it
isn't critical. It just seemed like something that could be included
by default.

Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

lower/uppercase rewrite maps

[Chris Cheshire]

Looking through the documentation for the rewrite valve [1], I see
there is an example of how to write and use a rewrite map to convert a
value to upper case. This is the inverse of what I want (lowercase),
so great, easy enough to implement. This seems like something that
could be included by default but I couldn't see anything in
catalina.jar.

Is this something that would be included if I create a patch for it,
and how would I go about it?

Chris

[1] http://tomcat.apache.org/tomcat-9.0-doc/rewrite.html

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: remote jmx monitoring through ssh tunnel

[Chris Cheshire]

On Thu, Dec 12, 2019 at 10:05 AM Christopher Schultz
 wrote:
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris,
>
> On 12/11/19 15:52, Chris Cheshire wrote:
> > On Wed, Dec 11, 2019 at 12:24 PM Christopher Schultz
> >  wrote:
> >>
> >> -BEGIN PGP SIGNED MESSAGE- Hash: SHA256
> >>
> >>
> >>
> >> On 12/10/19 12:59, Chris Cheshire wrote:
> >>> On Tue, Dec 10, 2019 at 11:58 AM Chris Cheshire
> >>>  wrote:
> >>>>
> >>>> On Tue, Dec 10, 2019 at 9:42 AM Christopher Schultz
> >>>>  wrote:
> >>>>>
> >>>>> -BEGIN PGP SIGNED MESSAGE- Hash: SHA256
> >>>>>
> >>>>> Chris,
> >>>>>
> >>>>> On 12/9/19 17:10, Chris Cheshire wrote:
> >>>>>> In CATALINA_BASE/bin/setenv.sh I have the following :
> >>>>>>
> >>>>>> CATALINA_OPTS="-Dcom.sun.management.jmxremote
> >>>>>> -Dcom.sun.management.jmxremote.ssl=false
> >>>>>> -Dcom.sun.management.jmxremote.authenticate=false"
> >>>>>
> >>>>> Okay.
> >>>>>
> >>>>>> In CATALINA_BASE/conf/server.xml I have a listener
> >>>>>> configured :
> >>>>>>
> >>>>>>  >>>>>> className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener"
> >>>>>>
> >>>>>>
> >>
> >>>>>>
> rmiRegistryPortPlatform="10001" rmiServerPortPlatform="10002"
> >>>>>> useLocalPorts="true" />
> >>>>>>
> >>>>>>
> >>>>>> Upon startup I see in logs : INFO [main]
> >>>>>> org.apache.catalina.mbeans.JmxRemoteLifecycleListener.createServe
> r
> >>>>>>
> >>>>>>
> >>
> >>>>>>
> The JMX Remote Listener has configured the registry on port
> >>>>>> [10001] and the server on port [10002] for the
> >>>>>> [Platform] server
> >>>>>>
> >>>>>>
> >>>>>> $ netstat -an | grep 10001 tcp4   0  0
> >>>>>> 127.0.0.1.10001 *.*LISTEN tcp6
> >>>>>> 0 0  ::1.10001 *.*LISTEN
> >>>>>>
> >>>>>> On my local machine I have a tunnel set up as follows :
> >>>>>> ssh -N -L10001:localhost:10001 -L10002:localhost:10002
> >>>>>> user@remotehost
> >>>>>>
> >>>>>> (where user is the user tomcat is running under)
> >>>>>>
> >>>>>> When I try to add a remote JMX connection in VisualVM on
> >>>>>> my client machine to localhost:10001 I get an error
> >>>>>> dialog after a brief delay with the message "Cannot
> >>>>>> connect to localhost:10001 using
> >>>>>> service:jmx:rmi:///jndi/rmi://localhost:10001/jmxrmi". If
> >>>>>> I change it to port 10002 I get the same error. On the
> >>>>>> server at this time : $ netstat -an | grep 10001 tcp4
> >>>>>> 0  0 127.0.0.1.10001 *.*LISTEN
> >>>>>> tcp6   0 0  ::1.10001 *.*LISTEN
> >>>>>> tcp4   0 0  127.0.0.1.62637 127.0.0.1.10001
> >>>>>> TIME_WAIT
> >>>>>>
> >>>>>>
> >>>>>> If I try to use jconsole connecting to port 10001 I get
> >>>>>> the error "Connection failed: non-JRMP server at remote
> >>>>>> endpoint". Connecting to port 10002 I get the error
> >>>>>> "Connection failed: no such object in table"
> >>>>>
> >>>>> You should be using the port defined by
> >>>>> rmiRegistryPortPlatform, so 10001 is the correct port to
> >>>>> use.
> >>>>>
> >>>>>> I've been through the tomcat configuration documentation
> >>>>>> a couple times but I can't see what else I need to
> >>>>>> configure.
> >>>>>
> >>>>> What you have looks good to me without reproducing it
> >>>>> myself. Can you do :
> >>

Re: remote jmx monitoring through ssh tunnel

[Chris Cheshire]

On Fri, Dec 13, 2019 at 3:52 AM Jäkel, Guido  wrote:
>
> Dear all,
>
> some time ago, I also need to pass JXM through a tunneled connection (using 
> STunnel). The Problem with JMX via RMI is that here is more than one 
> connection and involved in the handshake is an IP address that is different 
> on both ends of the tunnel. A solution was to use an alternative transport 
> layer named JMXMP because this need only one connection.
>
> You may use my notes on https://github.com/gjaekel/jmxmp-lifecycle-listener 
> as a starting point.
>
> One have to provide additionals JARS, on Tomcat to $CATALINA_HOME/lib. Then 
> one have to compile and install an additional listener
>
>  className="javax.management.remote.extension.JMXMPLifecycleListener" 
> port="" />
>
> The client also must be "undergrid" with an additional library, e.g.
>
> visualvm --cp:a jmxremote_optional.jar
>
> and the connection URL must use the alternative protocol
>
> service:jmx:jmxmp://:
>
>
> with greetings
>
> Guido
>
>
Guido,

Thanks for the pointers. I will look into this soon.

Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: remote jmx monitoring through ssh tunnel

[Chris Cheshire]

On Wed, Dec 11, 2019 at 12:24 PM Christopher Schultz
 wrote:
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
>
>
> On 12/10/19 12:59, Chris Cheshire wrote:
> > On Tue, Dec 10, 2019 at 11:58 AM Chris Cheshire
> >  wrote:
> >>
> >> On Tue, Dec 10, 2019 at 9:42 AM Christopher Schultz
> >>  wrote:
> >>>
> >>> -BEGIN PGP SIGNED MESSAGE- Hash: SHA256
> >>>
> >>> Chris,
> >>>
> >>> On 12/9/19 17:10, Chris Cheshire wrote:
> >>>> In CATALINA_BASE/bin/setenv.sh I have the following :
> >>>>
> >>>> CATALINA_OPTS="-Dcom.sun.management.jmxremote
> >>>> -Dcom.sun.management.jmxremote.ssl=false
> >>>> -Dcom.sun.management.jmxremote.authenticate=false"
> >>>
> >>> Okay.
> >>>
> >>>> In CATALINA_BASE/conf/server.xml I have a listener configured
> >>>> :
> >>>>
> >>>>  >>>> className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener"
> >>>>
> >>>>
> rmiRegistryPortPlatform="10001" rmiServerPortPlatform="10002"
> >>>> useLocalPorts="true" />
> >>>>
> >>>>
> >>>> Upon startup I see in logs : INFO [main]
> >>>> org.apache.catalina.mbeans.JmxRemoteLifecycleListener.createServer
> >>>>
> >>>>
> The JMX Remote Listener has configured the registry on port
> >>>> [10001] and the server on port [10002] for the [Platform]
> >>>> server
> >>>>
> >>>>
> >>>> $ netstat -an | grep 10001 tcp4   0  0
> >>>> 127.0.0.1.10001 *.*LISTEN tcp6   0
> >>>> 0  ::1.10001 *.*LISTEN
> >>>>
> >>>> On my local machine I have a tunnel set up as follows : ssh
> >>>> -N -L10001:localhost:10001 -L10002:localhost:10002
> >>>> user@remotehost
> >>>>
> >>>> (where user is the user tomcat is running under)
> >>>>
> >>>> When I try to add a remote JMX connection in VisualVM on my
> >>>> client machine to localhost:10001 I get an error dialog after
> >>>> a brief delay with the message "Cannot connect to
> >>>> localhost:10001 using
> >>>> service:jmx:rmi:///jndi/rmi://localhost:10001/jmxrmi". If I
> >>>> change it to port 10002 I get the same error. On the server
> >>>> at this time : $ netstat -an | grep 10001 tcp4   0  0
> >>>> 127.0.0.1.10001 *.*LISTEN tcp6   0
> >>>> 0  ::1.10001 *.*LISTEN tcp4   0
> >>>> 0  127.0.0.1.62637 127.0.0.1.10001TIME_WAIT
> >>>>
> >>>>
> >>>> If I try to use jconsole connecting to port 10001 I get the
> >>>> error "Connection failed: non-JRMP server at remote
> >>>> endpoint". Connecting to port 10002 I get the error
> >>>> "Connection failed: no such object in table"
> >>>
> >>> You should be using the port defined by
> >>> rmiRegistryPortPlatform, so 10001 is the correct port to use.
> >>>
> >>>> I've been through the tomcat configuration documentation a
> >>>> couple times but I can't see what else I need to configure.
> >>>
> >>> What you have looks good to me without reproducing it myself.
> >>> Can you do :
> >>>
> >>> $ netstat -an | grep 1000[0-9]
> >>>
> >>> ?
> >>>
> >>> Just to be sure about both ports?
> >>>
> >>
> >> $ netstat -an | grep 1000[0-9] tcp6   0  0 :::10001
> >> :::*LISTEN tcp6   0  0 :::10002
> >> :::*LISTEN
> >>
> >>
> >> H. Tomcat is only listening on ipv6 ports, but my tunnel is
> >> using ipv4. After digging around [1], I added this to
> >> CATALINA_OPTS in setenv.sh
> >>
> >> -Djava.net.preferIPv4Stack=true
> >> -Djava.net.preferIPv4Addresses=true
> >>
> >> $ netstat -an | grep 1000[0-9] tcp0  0 0.0.0.0:10001
> >> 0.0.0.0:*   LISTEN tcp0  0 0.0.0.0:10002
> >> 0.0.0.0:*   LISTEN
> >>
> >> When I try to connect with jconsole I get th

Re: remote jmx monitoring through ssh tunnel

[Chris Cheshire]

Peter,

On Wed, Dec 11, 2019 at 2:38 AM Peter Kreuser  wrote:
>
> Chris‘,
>
> > Am 10.12.2019 um 18:59 schrieb Chris Cheshire :
> >
> > On Tue, Dec 10, 2019 at 11:58 AM Chris Cheshire  
> > wrote:
> >>
> >>> On Tue, Dec 10, 2019 at 9:42 AM Christopher Schultz
> >>>  wrote:
> >>>
> >>> -BEGIN PGP SIGNED MESSAGE-
> >>> Hash: SHA256
> >>>
> >>> Chris,
> >>>
> >>> On 12/9/19 17:10, Chris Cheshire wrote:
> >>>> In CATALINA_BASE/bin/setenv.sh I have the following :
> >>>>
> >>>> CATALINA_OPTS="-Dcom.sun.management.jmxremote
> >>>> -Dcom.sun.management.jmxremote.ssl=false
> >>>> -Dcom.sun.management.jmxremote.authenticate=false"
> >>>
> >>> Okay.
> >>>
> >>>> In CATALINA_BASE/conf/server.xml I have a listener configured :
> >>>>
> >>>>  >>>> className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener"
> >>>> rmiRegistryPortPlatform="10001" rmiServerPortPlatform="10002"
> >>>> useLocalPorts="true" />
> >>>>
> >>>>
> >>>> Upon startup I see in logs : INFO [main]
> >>>> org.apache.catalina.mbeans.JmxRemoteLifecycleListener.createServer
> >>>> The JMX Remote Listener has configured the registry on port
> >>>> [10001] and the server on port [10002] for the [Platform] server
> >>>>
> >>>>
>
> I didn‘t read it anywhere. Did you add the catalina-jmx.jar to the classpath?
>
> Peter
>

What is that jar and where does it come from? I don't see it in the TC
9 distribution, nor can I find mention of it in the docs [1] [2]

Chris

[1] https://tomcat.apache.org/tomcat-9.0-doc/monitoring.html
[2] https://tomcat.apache.org/tomcat-9.0-doc/config/listeners.html

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: remote jmx monitoring through ssh tunnel

[Chris Cheshire]

On Tue, Dec 10, 2019 at 11:58 AM Chris Cheshire  wrote:
>
> On Tue, Dec 10, 2019 at 9:42 AM Christopher Schultz
>  wrote:
> >
> > -BEGIN PGP SIGNED MESSAGE-
> > Hash: SHA256
> >
> > Chris,
> >
> > On 12/9/19 17:10, Chris Cheshire wrote:
> > > In CATALINA_BASE/bin/setenv.sh I have the following :
> > >
> > > CATALINA_OPTS="-Dcom.sun.management.jmxremote
> > > -Dcom.sun.management.jmxremote.ssl=false
> > > -Dcom.sun.management.jmxremote.authenticate=false"
> >
> > Okay.
> >
> > > In CATALINA_BASE/conf/server.xml I have a listener configured :
> > >
> > >  > > className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener"
> > > rmiRegistryPortPlatform="10001" rmiServerPortPlatform="10002"
> > > useLocalPorts="true" />
> > >
> > >
> > > Upon startup I see in logs : INFO [main]
> > > org.apache.catalina.mbeans.JmxRemoteLifecycleListener.createServer
> > > The JMX Remote Listener has configured the registry on port
> > > [10001] and the server on port [10002] for the [Platform] server
> > >
> > >
> > > $ netstat -an | grep 10001 tcp4   0  0  127.0.0.1.10001
> > > *.*LISTEN tcp6   0  0  ::1.10001
> > > *.*LISTEN
> > >
> > > On my local machine I have a tunnel set up as follows : ssh -N
> > > -L10001:localhost:10001 -L10002:localhost:10002 user@remotehost
> > >
> > > (where user is the user tomcat is running under)
> > >
> > > When I try to add a remote JMX connection in VisualVM on my client
> > > machine to localhost:10001 I get an error dialog after a brief
> > > delay with the message "Cannot connect to localhost:10001 using
> > > service:jmx:rmi:///jndi/rmi://localhost:10001/jmxrmi". If I change
> > > it to port 10002 I get the same error. On the server at this time
> > > : $ netstat -an | grep 10001 tcp4   0  0  127.0.0.1.10001
> > > *.*LISTEN tcp6   0  0  ::1.10001
> > > *.*LISTEN tcp4   0  0  127.0.0.1.62637
> > > 127.0.0.1.10001TIME_WAIT
> > >
> > >
> > > If I try to use jconsole connecting to port 10001 I get the error
> > > "Connection failed: non-JRMP server at remote endpoint". Connecting
> > > to port 10002 I get the error "Connection failed: no such object
> > > in table"
> >
> > You should be using the port defined by rmiRegistryPortPlatform, so
> > 10001 is the correct port to use.
> >
> > > I've been through the tomcat configuration documentation a couple
> > > times but I can't see what else I need to configure.
> >
> > What you have looks good to me without reproducing it myself. Can you do
> > :
> >
> > $ netstat -an | grep 1000[0-9]
> >
> > ?
> >
> > Just to be sure about both ports?
> >
>
> $ netstat -an | grep 1000[0-9]
> tcp6   0  0 :::10001:::*LISTEN
> tcp6   0  0 :::10002:::*LISTEN
>
>
> H. Tomcat is only listening on ipv6 ports, but my tunnel is using
> ipv4. After digging around [1], I added this to CATALINA_OPTS in
> setenv.sh
>
> -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Addresses=true
>
> $ netstat -an | grep 1000[0-9]
> tcp0  0 0.0.0.0:10001   0.0.0.0:*   LISTEN
> tcp0  0 0.0.0.0:10002   0.0.0.0:*   LISTEN
>
> When I try to connect with jconsole I get the same error (non-JRMP
> server at remote endpoint), with the server showing
>
> tcp0  0 0.0.0.0:10001   0.0.0.0:*   LISTEN
> tcp0  0 0.0.0.0:10002   0.0.0.0:*   LISTEN
> tcp0  0 127.0.0.1:10001 127.0.0.1:43803 TIME_WAIT
> tcp0  0 127.0.0.1:10001 127.0.0.1:43815 TIME_WAIT
>
>
> I have also updated sshd_config with
>
> PermitTunnel yes
>
> and restarted that. Still no change.
>
> Chris
>
>
> [1] 
> https://serverfault.com/questions/390840/how-does-one-get-tomcat-to-bind-to-ipv4-address


As a followup to take the tunnel out of the equation I downloaded
jmxterm [1] on the server and tried to connect


$ java -jar jmxterm-1.0.0-uber.jar
Welcome to JMX terminal. Type "help" for available commands.
$>open localhost:10001
#RuntimeIOException: Runtime IO exception: Failed to r

Re: remote jmx monitoring through ssh tunnel

[Chris Cheshire]

On Tue, Dec 10, 2019 at 9:42 AM Christopher Schultz
 wrote:
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris,
>
> On 12/9/19 17:10, Chris Cheshire wrote:
> > In CATALINA_BASE/bin/setenv.sh I have the following :
> >
> > CATALINA_OPTS="-Dcom.sun.management.jmxremote
> > -Dcom.sun.management.jmxremote.ssl=false
> > -Dcom.sun.management.jmxremote.authenticate=false"
>
> Okay.
>
> > In CATALINA_BASE/conf/server.xml I have a listener configured :
> >
> >  > className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener"
> > rmiRegistryPortPlatform="10001" rmiServerPortPlatform="10002"
> > useLocalPorts="true" />
> >
> >
> > Upon startup I see in logs : INFO [main]
> > org.apache.catalina.mbeans.JmxRemoteLifecycleListener.createServer
> > The JMX Remote Listener has configured the registry on port
> > [10001] and the server on port [10002] for the [Platform] server
> >
> >
> > $ netstat -an | grep 10001 tcp4   0  0  127.0.0.1.10001
> > *.*LISTEN tcp6   0  0  ::1.10001
> > *.*LISTEN
> >
> > On my local machine I have a tunnel set up as follows : ssh -N
> > -L10001:localhost:10001 -L10002:localhost:10002 user@remotehost
> >
> > (where user is the user tomcat is running under)
> >
> > When I try to add a remote JMX connection in VisualVM on my client
> > machine to localhost:10001 I get an error dialog after a brief
> > delay with the message "Cannot connect to localhost:10001 using
> > service:jmx:rmi:///jndi/rmi://localhost:10001/jmxrmi". If I change
> > it to port 10002 I get the same error. On the server at this time
> > : $ netstat -an | grep 10001 tcp4   0  0  127.0.0.1.10001
> > *.*LISTEN tcp6   0  0  ::1.10001
> > *.*LISTEN tcp4   0  0  127.0.0.1.62637
> > 127.0.0.1.10001TIME_WAIT
> >
> >
> > If I try to use jconsole connecting to port 10001 I get the error
> > "Connection failed: non-JRMP server at remote endpoint". Connecting
> > to port 10002 I get the error "Connection failed: no such object
> > in table"
>
> You should be using the port defined by rmiRegistryPortPlatform, so
> 10001 is the correct port to use.
>
> > I've been through the tomcat configuration documentation a couple
> > times but I can't see what else I need to configure.
>
> What you have looks good to me without reproducing it myself. Can you do
> :
>
> $ netstat -an | grep 1000[0-9]
>
> ?
>
> Just to be sure about both ports?
>

$ netstat -an | grep 1000[0-9]
tcp6   0  0 :::10001:::*LISTEN
tcp6   0  0 :::10002:::*LISTEN


H. Tomcat is only listening on ipv6 ports, but my tunnel is using
ipv4. After digging around [1], I added this to CATALINA_OPTS in
setenv.sh

-Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Addresses=true

$ netstat -an | grep 1000[0-9]
tcp0  0 0.0.0.0:10001   0.0.0.0:*   LISTEN
tcp0  0 0.0.0.0:10002   0.0.0.0:*   LISTEN

When I try to connect with jconsole I get the same error (non-JRMP
server at remote endpoint), with the server showing

tcp0  0 0.0.0.0:10001   0.0.0.0:*   LISTEN
tcp0  0 0.0.0.0:10002   0.0.0.0:*   LISTEN
tcp0  0 127.0.0.1:10001 127.0.0.1:43803 TIME_WAIT
tcp0  0 127.0.0.1:10001 127.0.0.1:43815 TIME_WAIT


I have also updated sshd_config with

PermitTunnel yes

and restarted that. Still no change.

Chris


[1] 
https://serverfault.com/questions/390840/how-does-one-get-tomcat-to-bind-to-ipv4-address

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

remote jmx monitoring through ssh tunnel

[Chris Cheshire]

Server : Debian 8, Tomcat 9.0.29, OpenJDK 1.8
Client : MacOS Mojave

After reading a recent thread here on monitoring database connections
via JMX I am trying to set it up on a sandbox. I would prefer to use
an SSH tunnel to connect than open up ports on the firewall if
possible.

In CATALINA_BASE/bin/setenv.sh I have the following :

CATALINA_OPTS="-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false"

In CATALINA_BASE/conf/server.xml I have a listener configured :

  


Upon startup I see in logs :
INFO [main] org.apache.catalina.mbeans.JmxRemoteLifecycleListener.createServer
The JMX Remote Listener has configured the registry on port [10001]
and the server on port [10002] for the [Platform] server


$ netstat -an | grep 10001
tcp4   0  0  127.0.0.1.10001*.*LISTEN
tcp6   0  0  ::1.10001  *.*LISTEN

On my local machine I have a tunnel set up as follows :
ssh -N -L10001:localhost:10001 -L10002:localhost:10002 user@remotehost

(where user is the user tomcat is running under)

When I try to add a remote JMX connection in VisualVM on my client
machine to localhost:10001 I get an error dialog after a brief delay
with the message "Cannot connect to localhost:10001 using
service:jmx:rmi:///jndi/rmi://localhost:10001/jmxrmi".
If I change it to port 10002 I get the same error. On the server at this time :
$ netstat -an | grep 10001
tcp4   0  0  127.0.0.1.10001*.*LISTEN
tcp6   0  0  ::1.10001  *.*LISTEN
tcp4   0  0  127.0.0.1.62637127.0.0.1.10001TIME_WAIT


If I try to use jconsole connecting to port 10001 I get the error
"Connection failed: non-JRMP server at remote endpoint". Connecting to
port 10002 I get the error "Connection failed: no such object in
table"

I've been through the tomcat configuration documentation a couple
times but I can't see what else I need to configure.

Any suggestions?

Thanks

Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: problem running digest.sh

[Chris Cheshire]

> On Dec 2, 2019, at 4:21 PM, Mark Thomas  wrote:
> 
> On 02/12/2019 21:07, Chris Cheshire wrote:
> 
> 
> 
>> File on bugzilla?
> 
> No need. It is https://bz.apache.org/bugzilla/show_bug.cgi?id=63815
> 
> It is fixed as far as we can until the jdg issue is fixed.
> 
> Mark


My search-fu is not very good, I didn’t find anything when I looked first.

Cheers :)
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

problem running digest.sh

[Chris Cheshire]

tomcat 9.0.29, Debian 8, openjdk 1.8.0_232-b09

Trying to run digest.sh :

$ cd /opt/apache-tomcat/9.0.29/bin
$ ./digest.sh
Error: Could not find or load main class
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
$ CATALINA_HOME=/opt/apache-tomcat-9.0.29 ./digest.sh
Error: Could not find or load main class
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager


Looks like it is confusing java opts with the main class somehow.
Digging through the shell scripts and comparing to previous versions I
have floating around I found this :

$ diff tool-wrapper.sh /opt/apache-tomcat-9.0.19/bin/tool-wrapper.sh
149c149
< exec "$_RUNJAVA" "$JAVA_OPTS" $TOOL_OPTS \
---
> exec "$_RUNJAVA" $JAVA_OPTS $TOOL_OPTS \

If I unquote JAVA_OPTS it works properly :

$ ./digest.sh
Usage: RealmBase [-a ] [-e ] [-i ]
[-s ] [-k ] [-h ]


I don't know enough about the scripts though to know whether the
unquoting is breaking anything else. It didn't seem to affect
start/stop via catalina.sh however.

File on bugzilla?

Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: catalina_out env var

[Chris Cheshire]

On Fri, Nov 1, 2019 at 1:59 PM Mark Thomas  wrote:
>
> On November 1, 2019 3:25:36 PM UTC, Chris Cheshire  
> wrote:
> >(Tomcat 9, Debian & Centos Unix)
> >
> >I use systemd to start tomcat with base and home separated using
> >catalina.sh. For the service script, values for JAVA_HOME,
> >CATALINA_HOME, CATALINA_BASE and CATALINA_OUT are defined there.
> >
> >Inside catalina.sh it says this about CATALINA_OUT
> >
> >#   CATALINA_OUT(Optional) Full path to a file where stdout and
> >stderr
> >#   will be redirected.
> >#   Default is $CATALINA_BASE/logs/catalina.out
> >
> >If this isn't defined, catalina.sh sets a default. It seems that
> >everything that goes in here also goes in catalina.date.log
> >(controlled by the tomcat's logging.properties), so why does it exist?
> >Can I force a redirection to /dev/null instead so I don't have
> >duplicated logs?
> >
>
> You can control this in logging.properties  Just disable the console logger.
>
> You don't want redirect to /dev/null as then you will be unable to take 
> thread dumps or anything else that writes to stdout.
>
> It is setup this way for users running Tomcat directly from the console.
>
> Mark
>

OK so I just remove java.util.logging.ConsoleHandler from the
.handlers line? Does this affect the content of catalina.log, or
catalina.date.log?

I'm less concerned about the chattiness than the redundancy. I've got
two logs that contain the same thing, one I have to control with
logrotate.d and the other automatically rotates but is never cleared
so I also need a cron job to remove old logs. I'm just trying to clean
things up.

Thanks

Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

catalina_out env var

[Chris Cheshire]

(Tomcat 9, Debian & Centos Unix)

I use systemd to start tomcat with base and home separated using
catalina.sh. For the service script, values for JAVA_HOME,
CATALINA_HOME, CATALINA_BASE and CATALINA_OUT are defined there.

Inside catalina.sh it says this about CATALINA_OUT

#   CATALINA_OUT(Optional) Full path to a file where stdout and stderr
#   will be redirected.
#   Default is $CATALINA_BASE/logs/catalina.out

If this isn't defined, catalina.sh sets a default. It seems that
everything that goes in here also goes in catalina.date.log
(controlled by the tomcat's logging.properties), so why does it exist?
Can I force a redirection to /dev/null instead so I don't have
duplicated logs?

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Redirecting a naked domain to sub domain

[Chris Cheshire]

On Thu, Aug 15, 2019 at 2:54 PM Simon Funnell  wrote:
>
> On Thu, 15 Aug 2019 at 01:00, Chris Cheshire  wrote:
>
> > On Wed, Aug 14, 2019 at 5:18 PM Simon Funnell 
> > wrote:
> > >
> > > On Tue, 6 Aug 2019 at 14:33, John Larsen 
> > wrote:
> > >
> > > > I usually handle the redirect on the webserver side and set the alias
> > in
> > > > server.xml to the subdomain.
> > > >
> > > > John Larsen
> > > >
> > > >
> > > >
> > > I have now successfully loaded the rewrite valve in the application
> > > context.xml but I am getting the following error on start up:
> > > java.util.regex.PatternSyntaxException
> > >
> > > This is the rewrite.config file:
> > >
> > > RewriteCond %{HTTP_HOST} !^(.*)\.domain\.org\ [NC]
> > > RewriteRule ^(.*)$ https://www.domain.org/$1 [R=301,L]
> > >
> > > It looks like the first line is an error and I don't know enough about
> > > regex to fix it.
> > >
> > > Any help is great thanks.
> > >
> > >
> >
> > The \ after .org is misplaced (it's trying to quote the space
> > following it), remove that and try again
> >
> > Chris
> >
> >
> > Chris
> >
> >
> It kind of works now but when it redirects it adds two slashes to the end
> of domain:
>
> https://www.domain.org//
>
> which is causing an internal server error for the URL not br normalised
>
>
>

That's odd. I do the inverse of that to remove www and have a naked
domain, but using mod_rewrite in httpd. The rewrite rule is otherwise
the same. I thought the syntax was the same. Maybe something inside
tomcat is forcing a / at the end of the domain for canonicalization,
and then the extra / is getting added by the rewrite valve rule?

Looking at a different server where I force the www, I have this instead :

RewriteRule ^/(.*) https://www.domain.com%{REQUEST_URI} [L,R=301]

You can try this instead. It won't be putting a / in there unless it
is already present in the uri passed (unlike the other version where a
/ will be appended even if the uri is blank)

HTH,

Chris


> > >
> > > >
> >
> > > > On Tue, Aug 6, 2019 at 7:11 AM Simon Funnell 
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I was going to do a redirect at my dns provider from the naked
> > domain to
> > > > > the www subdomain. Unfortunatly this doesn't really work if it is
> > over
> > > > > https so I am going to have to do the redirect at the host level. My
> > > > > question is, is this what the rewrite valve is for? I was going to
> > set up
> > > > > two hosts, one for the naked domain and one for the subdomain with a
> > set
> > > > up
> > > > > like the following:
> > > > >
> > > > >   > > > > unpackWARs="true" autoDeploy="true">
> > > > >  
> > > > >   
> > > > >
> > > > >  > > > > unpackWARs="true" autoDeploy="true">
> > > > >
> > > > > 
> > > > >  > > > > directory="logs"
> > > > >prefix="localhost_access_log" suffix=".txt"
> > > > >pattern="%h %l %u %t %r %s %b" />
> > > > >
> > > > >   
> > > > >
> > > > > If this is what the rewrite valve is for, how do I configure it to
> > > > redirect
> > > > > to the subdomain apart from when it accesses the .well-known folder
> > > > (which
> > > > > is needed to get a letsencrypt certificate).
> > > > >
> > > > > If the rewrite valve is not the correct way to do this do I just
> > deploy a
> > > > > catch all servlet or a filter to do the redirect instead?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Simon
> > > > >
> > > >
> >
> > -
> > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> > For additional commands, e-mail: users-h...@tomcat.apache.org
> >
> >

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Redirecting a naked domain to sub domain

[Chris Cheshire]

On Wed, Aug 14, 2019 at 5:18 PM Simon Funnell  wrote:
>
> On Tue, 6 Aug 2019 at 14:33, John Larsen  wrote:
>
> > I usually handle the redirect on the webserver side and set the alias in
> > server.xml to the subdomain.
> >
> > John Larsen
> >
> >
> >
> I have now successfully loaded the rewrite valve in the application
> context.xml but I am getting the following error on start up:
> java.util.regex.PatternSyntaxException
>
> This is the rewrite.config file:
>
> RewriteCond %{HTTP_HOST} !^(.*)\.domain\.org\ [NC]
> RewriteRule ^(.*)$ https://www.domain.org/$1 [R=301,L]
>
> It looks like the first line is an error and I don't know enough about
> regex to fix it.
>
> Any help is great thanks.
>
>

The \ after .org is misplaced (it's trying to quote the space
following it), remove that and try again

Chris


Chris

>
> >

> > On Tue, Aug 6, 2019 at 7:11 AM Simon Funnell 
> > wrote:
> >
> > > Hi,
> > >
> > > I was going to do a redirect at my dns provider from the naked domain to
> > > the www subdomain. Unfortunatly this doesn't really work if it is over
> > > https so I am going to have to do the redirect at the host level. My
> > > question is, is this what the rewrite valve is for? I was going to set up
> > > two hosts, one for the naked domain and one for the subdomain with a set
> > up
> > > like the following:
> > >
> > >   > > unpackWARs="true" autoDeploy="true">
> > >  
> > >   
> > >
> > >  > > unpackWARs="true" autoDeploy="true">
> > >
> > > 
> > >  > > directory="logs"
> > >prefix="localhost_access_log" suffix=".txt"
> > >pattern="%h %l %u %t %r %s %b" />
> > >
> > >   
> > >
> > > If this is what the rewrite valve is for, how do I configure it to
> > redirect
> > > to the subdomain apart from when it accesses the .well-known folder
> > (which
> > > is needed to get a letsencrypt certificate).
> > >
> > > If the rewrite valve is not the correct way to do this do I just deploy a
> > > catch all servlet or a filter to do the redirect instead?
> > >
> > > Thanks,
> > >
> > > Simon
> > >
> >

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

rewritevalve logging

[Chris Cheshire]

I am trying to convert some url rewriting rules from tuckey rewrite
filter to tomcat's rewrite valve.

In my context.xml I have



In my WEB-INF directory I have a rewrite.config file with a couple of rules.

In tomcat's logging.properties I have

org.apache.catalina.core.ContainerBase.[Catalina].[hostname].level = FINEST

in catalina.out I see

21-Jan-2019 19:28:36.969 FINE [main]
org.apache.catalina.valves.rewrite.RewriteValve.startInternal Read
configuration from: /WEB-INF/rewrite.config
21-Jan-2019 19:28:36.971 FINE [main]
org.apache.catalina.valves.rewrite.RewriteValve.parse Add rule with
pattern ^(.*)\.[0-9]{13}\.(css|js)$ and substitution $1.$

(plus other lines for other rules)

on startup but this is all I see.

The few rules I have in there work, but I would like to debug as I go
when adding more rules. Does the valve not log anything as it is
processing rules?

Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: lingering mysql connections

[Chris Cheshire]

This is a bit of a lead but it doesn't entirely solve it. It's only
cleaning up connections to one of the datasources, I still have
connections for 2 of them duplicated :(

If I get a chance I'll go digging some more based on the SO rabbit
hole, but I can mitigate the problem by restarting tomcat (since it's
really only a sandbox issue where I do a lot of redeploys)

Cheers

Chris

On Fri, Dec 14, 2018 at 3:00 AM Greg Huber  wrote:
>
> I resolved the same using this link
>
> https://stackoverflow.com/questions/11872316/tomcat-guice-jdbc-memory-leak
>
> I created the ContextFinalizer to cleanup on shut down.
>
> Also, I had loads of strange sql issues which were resolved by switching to
> maria db.
>
> Cheers Greg
>
> On Thu, 13 Dec 2018 at 20:51, Chris Cheshire  wrote:
>
> > Tomcat 9.0.12, Debian, MySQL Server 5.7.23, Connector/J 5.1.46
> >
> > I am trying to fix a lingering database connection problem. When I
> > reload a context via the tomcat manager, connections to the
> > datasources are not being released in mysql. They are still on the 30
> > second activity cycle from the eviction thread. I can see this via
> > 'show processlist' in the mysql client - the 'time' column resets at
> > 30, and each connection has unique process ids that I can track per
> > reload.
> >
> > I have tomcat home and base split (multiple instances of tomcat across
> > different users), with the connector/j jar in tomcat_base/lib.
> >
> > In my webapp's META-INF/context.xml I have 3 different datasources,
> > config, data, sched. All have configuration like :
> >
> >> auth="Container"
> > type="javax.sql.DataSource"
> > driverClassName="com.mysql.jdbc.Driver"
> >
> > url="jdbc:mysql://localhost:3306/$DBNAME$?useUnicode=truecharacterEncoding=utf8useSSL=false"
> > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
> > username="$USER$"
> > password="$PASSWORD$"
> > maxActive="2"
> > maxIdle="1"
> > minIdle="1"
> > initialSize="1"
> > maxWait="3"
> > removeAbandoned="true"
> > removeAbandonedTimeout="60"
> > logAbandoned="true"
> > validationQuery="/* ping */"
> > testOnBorrow="true"
> > testWhileIdle="true"
> > timeBetweenEvictionRunsMillis="3"
> > defaultAutoCommit="false"
> > defaultIsolation="READ_COMMITTED" />
> >
> > Connections are obtained via
> >
> > Connection dbConn = ((DataSource)new
> > InitialContext().lookup(resourceName)).getConnection()
> >
> > Connections are all closed via
> >
> > dbConn.close()
> >
> > (Simplified greatly, there's convenience methods with exception
> > handling in there)
> >
> >
> >
> > In contextDestroyed() of a ServletContextListener I am calling
> >
> > AbandonedConnectionCleanupThread.checkedShutdown();
> >
> > I have an initialization servlet that must be manually called before
> > the webapp is fully online - it is used to load encrypted
> > configuration from the conf datasource. It does not touch the data
> > datasource, only conf and sched by virtue of starting the quartz
> > scheduler which is configured to use this datasource.
> >
> > My observation are :
> > * It doesn't matter what order I declare the datasources, they are
> > always getting opened in the order sched, conf, data (judging by
> > increased thread/process ids in mysql).
> > * When I start tomcat, I get 3 open connections in mysql, 1 to each of
> > the databases referenced by the datasources. If I immediately reload
> > via the manager, all 3 connections are destroyed and 3 new ones are
> > opened.
> > * Once I call the initialization servlet, and subsequently reload the
> > web app via the manager, previous connections to conf and sched are
> > still open in mysql, as well as new ones
> > * If I access any part of the web app that uses the data datasource,
> > those connections now also linger.
> > * Once I stop tomcat (and the JVM) all lingering connections are
> > closed in mysql.
> > * If I put the connector/j jar in my WEB-INF/lib instead of
> > tomcat_base/lib, I get the following warning on reload/shutdown
> >
> >

lingering mysql connections

[Chris Cheshire]

Tomcat 9.0.12, Debian, MySQL Server 5.7.23, Connector/J 5.1.46

I am trying to fix a lingering database connection problem. When I
reload a context via the tomcat manager, connections to the
datasources are not being released in mysql. They are still on the 30
second activity cycle from the eviction thread. I can see this via
'show processlist' in the mysql client - the 'time' column resets at
30, and each connection has unique process ids that I can track per
reload.

I have tomcat home and base split (multiple instances of tomcat across
different users), with the connector/j jar in tomcat_base/lib.

In my webapp's META-INF/context.xml I have 3 different datasources,
config, data, sched. All have configuration like :

  

Connections are obtained via

Connection dbConn = ((DataSource)new
InitialContext().lookup(resourceName)).getConnection()

Connections are all closed via

dbConn.close()

(Simplified greatly, there's convenience methods with exception
handling in there)



In contextDestroyed() of a ServletContextListener I am calling

AbandonedConnectionCleanupThread.checkedShutdown();

I have an initialization servlet that must be manually called before
the webapp is fully online - it is used to load encrypted
configuration from the conf datasource. It does not touch the data
datasource, only conf and sched by virtue of starting the quartz
scheduler which is configured to use this datasource.

My observation are :
* It doesn't matter what order I declare the datasources, they are
always getting opened in the order sched, conf, data (judging by
increased thread/process ids in mysql).
* When I start tomcat, I get 3 open connections in mysql, 1 to each of
the databases referenced by the datasources. If I immediately reload
via the manager, all 3 connections are destroyed and 3 new ones are
opened.
* Once I call the initialization servlet, and subsequently reload the
web app via the manager, previous connections to conf and sched are
still open in mysql, as well as new ones
* If I access any part of the web app that uses the data datasource,
those connections now also linger.
* Once I stop tomcat (and the JVM) all lingering connections are
closed in mysql.
* If I put the connector/j jar in my WEB-INF/lib instead of
tomcat_base/lib, I get the following warning on reload/shutdown

13-Dec-2018 20:19:53.968 WARNING [ajp-nio-8019-exec-3]
org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc
The web application [ct] registered the JDBC driver
[com.mysql.jdbc.Driver] but failed to unregister it when the web
application was stopped. To prevent a memory leak, the JDBC Driver has
been forcibly unregistered.

* There are no warnings or errors in catalina.out about abandoned
connections during runtime, reload or shutdown of the tomcat instance.
I have every connection being closed after use. (I have seen the
warnings when I have made a code mistake however, so the thread is
doing its job).
* If I remove the abandoned connection and eviction thread
configuration entirely, the connection activity timer resets on a 5
second cycle in mysql. If I explicitly change
timeBetweenEvictionRunsMillis to -1, the activity timer in mysql never
resets (which implies that the default is not -1 as the documentation
suggests)

So it seems that once a datasource is accessed, connections to it
perpetually linger in mysql until the JVM is shutdown.

I have had this issue for a long time, through tomcat 7, 8.5 and 9,
and multiple versions of mysql server and the connector., and also
using the commons pool. On my sandboxes I tend to reload a lot as I
deploy exploded, rather than deploy an entire WAR each time.

Googling led to multiple old bug reports filed w/ MySQL about the
cleanup thread, but nothing that would point to me doing anything
different than what I am doing. Are these lingering connections just a
fact of doing business with MySQL, or is there something I need to
configure/change in code to clean this up?


Thanks

Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Connection pool and parallel deployment problem

[Chris Cheshire]

On Mon, Nov 26, 2018 at 9:58 AM Gilles SCHLIENGER
 wrote:
>
> Hi,
> I understand your needs, but what is your problem, since you don't use 
> parallel deployment ?
> Your connections are not closed but they will not be recreated when you 
> deploy your webapp again, so there should be no problem ?
>
> Gilles
>

I added my 2c because I am seeing what you describe here

> > - I undeploy the first version of the webapp
> > - I check on my postgresql and mysql database and all connections are still 
> > opened (even if I wait for a long time)


Yes I know you are using parallel deployment and I am not, but that
doesn't necessarily mean that the problem you see is limited to only
parallel deployment. Maybe the problem can be simplified to "
Tomcat considers the connections closed, the database doesn't."

I could well be wrong, I'll leave it up to the gurus to decide :)


> -Message d'origine-
> De : Chris Cheshire [mailto:yahoono...@gmail.com]
> Envoyé : lundi 26 novembre 2018 15:27
> À : Tomcat Users List
> Objet : Re: Connection pool and parallel deployment problem
>
> I'm interested in what solution there is for this because I have the
> exact same problem but without parallel deployment.
>
> [snip]
>
> On Mon, Nov 26, 2018 at 3:54 AM Gilles SCHLIENGER
>  wrote:
> >
> > Hi Christopher,
> >
> > Thanks for your email.
> >
> > About connection pools not being closed, maybe the connection pool is 
> > closed but the connections to the database are not.
> >
> > Here are the tests I did:
> >
> > + TEST 1:
> > - I deploy my war
> > - I login to my webapp
> > - I check on my postgresql and mysql database that connections have been 
> > opened (select...)
> > - I undeploy my webapp through the manager webapp
> > - I check that the connections are still opened
> > - Tomcat is still running
> > - I redeploy the exact samed webapp and login to my application
> > - No other connection is being opened
> >
> > + TEST 2:
> > - I deploy my war
> > - I login to my webapp
> > - I check on my postgresql and mysql database that connections have been 
> > opened (select...)
> > - I deploy a new version of my application and login with another browser
> > - I check on my postgresql and mysql database that new connexions have been 
> > opened (they doubled)
> > - I undeploy the first version of the webapp
> > - I check on my postgresql and mysql database and all connections are still 
> > opened (even if I wait for a long time)
> >
> > For tests 1 and 2, I used C3p0, DBCP2 and even HikariCP
> >
>
>
> My results are the same. Using mysql, connection pools defined in
> server.xml (for user realm for access control to host manager app) and
> in my context.xml for my application. When I use the host-manager to
> reload an app, the connections are closed (no abandoned connection
> warnings) but not released. It is no until I stop tomcat completely
> and restart it that the connections are released in mysql. This has
> been the case for tomcat 7, 8.5 and 9 versions, with constant updates
> of mysql 5.7 and its driver, using both the apache connection pool and
> the tomcat connection pool. The driver lives in the tomcat/lib
> directory (since it is needed for the user realm datasource).
>
>
> > -Message d'origine-
> > De : Christopher Schultz [mailto:ch...@christopherschultz.net]
> > Envoyé : samedi 24 novembre 2018 17:19
> > À : users@tomcat.apache.org
> > Objet : Re: Connection pool and parallel deployment problem
> >
> > -BEGIN PGP SIGNED MESSAGE-
> > Hash: SHA256
> >
> > Gilles,
> >
> > On 11/23/18 05:07, Gilles SCHLIENGER wrote:
>
> [snip]
>
> >
> > > The warning/error messages are not actually linked to the
> > > connexions not closed. To ovoid these messages, you can: - move the
> > > jars (connexion pool, drivers...) into TOMCAT/lib - Have a
> > > ServletContextListener that calls
> > > AbandonedConnectionCleanupThread.checkedShutdown();
> >
> > The Connector/J devs haven't been able to understand how ClassLoaders
> > work, and have never really fixed that long-standing bug in a
> > satisfying way as far as I know. But you should always use a
> > ServletContextListener to attempt to shut-down the
> > AbadonedConnectionCleanuopThread.
> >
> > BTW that thread does not work as documented. It's claimed that it
> > doesn't start unless you issue a query with a timeout, but it always
> > starts whether or not you use queries with timeouts. It's frustrating.
> >
>
> Should we be using checked or unchecked shutdown?
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Connection pool and parallel deployment problem

[Chris Cheshire]

I'm interested in what solution there is for this because I have the
exact same problem but without parallel deployment.

[snip]

On Mon, Nov 26, 2018 at 3:54 AM Gilles SCHLIENGER
 wrote:
>
> Hi Christopher,
>
> Thanks for your email.
>
> About connection pools not being closed, maybe the connection pool is closed 
> but the connections to the database are not.
>
> Here are the tests I did:
>
> + TEST 1:
> - I deploy my war
> - I login to my webapp
> - I check on my postgresql and mysql database that connections have been 
> opened (select...)
> - I undeploy my webapp through the manager webapp
> - I check that the connections are still opened
> - Tomcat is still running
> - I redeploy the exact samed webapp and login to my application
> - No other connection is being opened
>
> + TEST 2:
> - I deploy my war
> - I login to my webapp
> - I check on my postgresql and mysql database that connections have been 
> opened (select...)
> - I deploy a new version of my application and login with another browser
> - I check on my postgresql and mysql database that new connexions have been 
> opened (they doubled)
> - I undeploy the first version of the webapp
> - I check on my postgresql and mysql database and all connections are still 
> opened (even if I wait for a long time)
>
> For tests 1 and 2, I used C3p0, DBCP2 and even HikariCP
>


My results are the same. Using mysql, connection pools defined in
server.xml (for user realm for access control to host manager app) and
in my context.xml for my application. When I use the host-manager to
reload an app, the connections are closed (no abandoned connection
warnings) but not released. It is no until I stop tomcat completely
and restart it that the connections are released in mysql. This has
been the case for tomcat 7, 8.5 and 9 versions, with constant updates
of mysql 5.7 and its driver, using both the apache connection pool and
the tomcat connection pool. The driver lives in the tomcat/lib
directory (since it is needed for the user realm datasource).


> -Message d'origine-
> De : Christopher Schultz [mailto:ch...@christopherschultz.net]
> Envoyé : samedi 24 novembre 2018 17:19
> À : users@tomcat.apache.org
> Objet : Re: Connection pool and parallel deployment problem
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Gilles,
>
> On 11/23/18 05:07, Gilles SCHLIENGER wrote:

[snip]

>
> > The warning/error messages are not actually linked to the
> > connexions not closed. To ovoid these messages, you can: - move the
> > jars (connexion pool, drivers...) into TOMCAT/lib - Have a
> > ServletContextListener that calls
> > AbandonedConnectionCleanupThread.checkedShutdown();
>
> The Connector/J devs haven't been able to understand how ClassLoaders
> work, and have never really fixed that long-standing bug in a
> satisfying way as far as I know. But you should always use a
> ServletContextListener to attempt to shut-down the
> AbadonedConnectionCleanuopThread.
>
> BTW that thread does not work as documented. It's claimed that it
> doesn't start unless you issue a query with a timeout, but it always
> starts whether or not you use queries with timeouts. It's frustrating.
>

Should we be using checked or unchecked shutdown?

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: best practice for temporary file

[Chris Cheshire]

James, this can be done using something like the following
configuration in your META-INF/context.xml (I use tomcat 8.5)


  


I use this to serve files at /images because they are dynamic content
and not part of the war itself. Make sure the directory specified by
"base" has read and write permission by the tomcat user at the OS
level and then you can serve files with a uri starting with /images/.

HTH

Chris

On Tue, Jul 17, 2018 at 10:21 AM James Finnall  wrote:
>
> Hello all,
>
> What is the best practice under Tomcat to create a data file and then
> allow the client to download it?
>
> So far all my attempts have failed.  If the Java app can write it then
> the client can not download, if client can download then the Java app
> can not write it.
>
> Does Tomcat have a provision to create a directory that be shared by
> both the app and the client?
>
>
> Thank you for your consideration,
> James
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: tomcat connection pool not closing properly

[Chris Cheshire]

On Thu, Apr 26, 2018 at 10:23 AM, Chris Cheshire <yahoono...@gmail.com> wrote:
[snip]

I've done some more testing with older and newer versions of tomcat,
and also swapping to use commons dbcp. All of them exhibit the same
behaviour.

So the question becomes is this expected behaviour (previous
connection pool instances lingering on webapp reload), or is it mysqld
misbehaving?

It's not a critical problem as restarting tomcat itself clears all the
connections, which isn't an issue on sandboxes and rare on the live
site anyway.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

tomcat connection pool not closing properly

[Chris Cheshire]

[ tomcat 8.5.30, mysql server 5.7.22, connector/j 5.1.46, centos 6, debian 8. ]

I recently upgraded my sandboxes from 8.5.24 to 8.5.28 and now .30 and
I have noticed that when a webapp is reloaded via the host manager,
the associated connection pool is not getting closed down properly in
the database.

I am not getting abandoned connection warnings in tomcat, rather mysql
eventually complains because its max connections has been exceeded. If
I run "show full processlist" in mysql after every reload, there are
now an extra 5 open connections for the webapp user. It seems that the
connection pools are still active, because the connection time column
cycles between 0 and timeBetweenEvictionRunsMillis (in seconds). The
connections themselves are not showing anything running on them
however.

Once I shutdown the tomcat process entirely, everything closes and
there are no errors in the tomcat logs about abandoned connections.

This wasn't the case sometime around 8.5.24. I'm not sure if this is a
problem in mysql or tomcat as I have upgraded mysql, the connector and
tomcat. I don't know where to look so I'll start here.

I have the following in my context.xml for my webapp's connection pool

  

I start tomcat via a systemd service unit.

# /etc/systemd/system/tomcat1.service
[Unit]
Description=Apache Tomcat Web Application Container (sandbox1)
Wants=network.target
After=syslog.target network.target

[Service]
Type=forking

EnvironmentFile=-/etc/default/tomcat1

ExecStart=/opt/apache-tomcat-8.5.30/bin/catalina.sh start
ExecStop=/opt/apache-tomcat-8.5.30/bin/catalina.sh stop

User=sandbox1
Group=sandbox1
UMask=0007

[Install]
WantedBy=multi-user.target

# /etc/default/tomcat1
JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
CATALINA_HOME=/opt/apache-tomcat-8.5.30
CATALINA_BASE=/home/sandbox1/tomcat
CATALINA_OUT=/home/sandbox1/tomcat/logs/catalina.out
CATALINA_PID=/home/sandbox1/tomcat/tomcat.pid

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: using default cacerts AND custom keystore

[Chris Cheshire]

On Fri, Feb 16, 2018 at 2:11 PM, Christopher Schultz
<ch...@christopherschultz.net> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris,
>
> On 2/14/18 3:34 PM, Chris Cheshire wrote:
>> On Wed, Feb 14, 2018 at 12:30 PM, Mark Thomas <ma...@apache.org>
>> wrote:
>>> On 14/02/18 17:17, Chris Cheshire wrote:
>>>> I am trying to set up my webapp to connect to an external
>>>> database via ssl. The database uses a self-signed certificate.
>>>> I have created a keystore with the self-signed CA and the
>>>> client key & cert. This keystore is configured via JAVA_OPTS in
>>>> setenv.sh
>>>>
>>>> JAVA_OPTS="-Djavax.net.ssl.keyStore=$CATALINA_BASE/conf/mysql.jks
>>>> \ -Djavax.net.ssl.keyStorePassword=password \
>>>> -Djavax.net.ssl.trustStore=$CATALINA_BASE/conf/mysql.jks \
>>>> -Djavax.net.ssl.trustStorePassword=password"
>>>>
>>>> This allows me to connect to the database without a problem.
>>>> However now I cannot connect to any external web service
>>>> because their certs will no longer validate.
>>>>
>>>> How do I configure tomcat such that the default cacerts is used
>>>> in addition to my self-signed certificates without importing
>>>> those into the default keystore (which is a Bad Idea™)?
>>>
>>> This is nothing to do with Tomcat. Tomcat plays no role in
>>> out-going TLS connections.
>>>
>>> The short answer is rather than using system properties, you
>>> should set the keystore and truststore programmatically so they
>>> apply just to the database connections rather than globally.
>>>
>>
>> So after a bit of digging [1,2] I found that this is achieved by
>> adding the following parameters to the mysql jdbc url in the
>> resource definition:
>>
>> clientCertificateKeyStoreUrl=file://${catalina.base}/conf/mysql.jks
>>
>>
> clientCertificateKeyStorePassword=password
>> trustCertificateKeyStoreUrl=file://${catalina.base}/conf/mysql.jks
>> trustCertificateKeyStorePassword=changeit
>>
>> Note that  [2] has a couple of errors. A) it specifies
>> clientCertificateKeyStore[Url|Password] in lieu of trustStore
>> system property, that should be
>> trustCertificateKeyStore[Url|Password] B) it specifies specifies
>> the urls in the form file:path_to_truststore_file, that is also
>> incorrect it should be file://path_to_truststore_file (which will
>> give a triple slash if an absolute path is used)
>>
>>
>> [1]
>> https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-con
> figuration-properties.html
>>
>>
> [2]
> https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-using
> - -ssl.html
>
> It might depend upon the version of Connector/J you are using. For
> example, I have this in my connection URL:
>
> '...=file:/etc/mysql/mysql.jks'
>
> Only a single leading / for an absolute path in my case, and it works
> as expected.
>
> The use of file:// was a historical mistake web browser users made,
> thinking that // was necessary between the protocol and anything after
> it. It was never the case, and any software requiring a URL like
> file:/// should be considered broken.
>
> - -chris

So I went back to retest everything to make sure I wasn't going crazy,
and it turns out that I actually am. It really is working as expected
without the double slash (and with). I guess I went crosseyed looking
at the error logs after so many attempts trying to get this working
initially.

Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: using default cacerts AND custom keystore

[Chris Cheshire]

On Wed, Feb 14, 2018 at 12:30 PM, Mark Thomas <ma...@apache.org> wrote:
> On 14/02/18 17:17, Chris Cheshire wrote:
>> I am trying to set up my webapp to connect to an external database via
>> ssl. The database uses a self-signed certificate. I have created a
>> keystore with the self-signed CA and the client key & cert. This
>> keystore is configured via JAVA_OPTS in setenv.sh
>>
>> JAVA_OPTS="-Djavax.net.ssl.keyStore=$CATALINA_BASE/conf/mysql.jks \
>> -Djavax.net.ssl.keyStorePassword=password \
>> -Djavax.net.ssl.trustStore=$CATALINA_BASE/conf/mysql.jks \
>> -Djavax.net.ssl.trustStorePassword=password"
>>
>> This allows me to connect to the database without a problem. However
>> now I cannot connect to any external web service because their certs
>> will no longer validate.
>>
>> How do I configure tomcat such that the default cacerts is used in
>> addition to my self-signed certificates without importing those into
>> the default keystore (which is a Bad Idea™)?
>
> This is nothing to do with Tomcat. Tomcat plays no role in out-going TLS
> connections.
>
> The short answer is rather than using system properties, you should set
> the keystore and truststore programmatically so they apply just to the
> database connections rather than globally.
>

So after a bit of digging [1,2] I found that this is achieved by
adding the following parameters to the mysql jdbc url in the resource
definition:

clientCertificateKeyStoreUrl=file://${catalina.base}/conf/mysql.jks
clientCertificateKeyStorePassword=password
trustCertificateKeyStoreUrl=file://${catalina.base}/conf/mysql.jks
trustCertificateKeyStorePassword=changeit

Note that  [2] has a couple of errors.
A) it specifies clientCertificateKeyStore[Url|Password] in lieu of
trustStore system property, that should be
trustCertificateKeyStore[Url|Password]
B) it specifies specifies the urls in the form
file:path_to_truststore_file, that is also incorrect it should be
file://path_to_truststore_file (which will give a triple slash if an
absolute path is used)


[1] 
https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html
[2] 
https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-using-ssl.html

Hope this helps someone else that happens to read the archives.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

using default cacerts AND custom keystore

[Chris Cheshire]

I am trying to set up my webapp to connect to an external database via
ssl. The database uses a self-signed certificate. I have created a
keystore with the self-signed CA and the client key & cert. This
keystore is configured via JAVA_OPTS in setenv.sh

JAVA_OPTS="-Djavax.net.ssl.keyStore=$CATALINA_BASE/conf/mysql.jks \
-Djavax.net.ssl.keyStorePassword=password \
-Djavax.net.ssl.trustStore=$CATALINA_BASE/conf/mysql.jks \
-Djavax.net.ssl.trustStorePassword=password"

This allows me to connect to the database without a problem. However
now I cannot connect to any external web service because their certs
will no longer validate.

How do I configure tomcat such that the default cacerts is used in
addition to my self-signed certificates without importing those into
the default keystore (which is a Bad Idea™)?

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: file permission problem SOLVED

[Chris Cheshire]

On Mon, Dec 18, 2017 at 1:11 PM, Chris Cheshire <yahoono...@gmail.com> wrote:
> Since moving from a yum controlled installation of tomcat and starting
> tomcat via jsvc I have a problem with permissions of uploaded files.
>
> Now all files that are written by tomcat have permissions of 0600,
> where they used to be 0644. I need the to be group readable, world
> readable is not necessary.
>
> I tried setting the UMASK environment variable in my script that
> starts jsvc but that has no effect.
>
> #!/bin/sh
>
> export UMASK=0027
> export JAVA_HOME="/usr/lib/jvm/java-1.8.0"
> export TOMCAT_USER="sandbox1"
> export CATALINA_HOME="/usr/local/apache-tomcat-8.5.24"
> export CATALINA_BASE="/home/sandbox1/tomcat"
> export CATALINA_OUT="/var/log/catalina-1.out"
> export CATALINA_PID="/var/run/tomcat-1.pid"
>
> $CATALINA_HOME/bin/daemon.sh $*
>
>
> Where and to what do I set the umask such that uploaded files have at
> least group read permission?

Solved it. After digging through daemon.sh, I see it is adding
$JSVC_OPTS to the call to jsvc.
Doing some googling led me to
https://issues.apache.org/jira/browse/DAEMON-221, which unfortunately
is not documented in jsvc -help (I'll see what I can do to submit a
patch for that).

Thus, it is a matter of adding

export JSVC_OPTS="-umask 0027"

to the above script.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

file permission problem

[Chris Cheshire]

Since moving from a yum controlled installation of tomcat and starting
tomcat via jsvc I have a problem with permissions of uploaded files.

Now all files that are written by tomcat have permissions of 0600,
where they used to be 0644. I need the to be group readable, world
readable is not necessary.

I tried setting the UMASK environment variable in my script that
starts jsvc but that has no effect.

#!/bin/sh

export UMASK=0027
export JAVA_HOME="/usr/lib/jvm/java-1.8.0"
export TOMCAT_USER="sandbox1"
export CATALINA_HOME="/usr/local/apache-tomcat-8.5.24"
export CATALINA_BASE="/home/sandbox1/tomcat"
export CATALINA_OUT="/var/log/catalina-1.out"
export CATALINA_PID="/var/run/tomcat-1.pid"

$CATALINA_HOME/bin/daemon.sh $*


Where and to what do I set the umask such that uploaded files have at
least group read permission?

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: context resources, replacement parameters

[Chris Cheshire]

I thought more about this over the weekend, and realised it is more
hassle than it is worth, and not because I don't want to do it. I
don't think it is actually feasible now that I think about it more.

Given that a JVM has one set of System.properties and one instance of
Tomcat can have multiple deployed contexts, I can't just inject a
single property. It would have to be mapped to some other value unique
to the context. Thus it is starting to look more like a "tail wagging
the dog" exercise, when I should just accept editing the config files
manually for now until I can get around to figuring ant out and
building them dynamically.

On Sat, Dec 9, 2017 at 11:19 AM, Mark Thomas <ma...@apache.org> wrote:
> On 08/12/17 22:13, Chris Cheshire wrote:
>> On Fri, Dec 8, 2017 at 3:36 PM, Mark Thomas <ma...@apache.org> wrote:
>>> On 08/12/17 18:49, Chris Cheshire wrote:
>>>> I have a directory resource set defined in my context.xml to handle images 
>>>> :
>>>>
>>>> 
>>>>   >>> base="${catalina.base}/cdn/p/images"
>>>> webAppMount="/images" />
>>>> 
>>>>
>>>> The /p in there actually represents the context path - a given sandbox
>>>> might have the same webapp deployed at different context paths
>>>> representing different development branches. Is it possible to use a
>>>> replacement parameter similar to catalina.base to replace the context
>>>> path the webapp is deployed at?
>>>
>>> Yes. You can use ant style property replacement in any XML file that is
>>> processed by the digester (server.xml, context.xml, web.xml)
>>>
>>> See the opening section of
>>>
>>> http://tomcat.apache.org/tomcat-9.0-doc/config/index.html
>>>
>>> for details.
>>>
>>> Mark
>>>
>>
>> I logged the System properties and there is nothing in there for the
>> context path.
>
> Sorry, I didn't quite understand what you were asking. This isn't
> possible out of the box.
>
> Injecting a special property (org.apache.tomcat.contextPath ?) should be
> doable when working with context specific configuration files -
> context.xml and web.xml - although there might be some edge cases with
> context.xml since sometimes the context path is set in the file. I think
> they can be handled with some documentation.
>
> Care to create an enhancement request? Better still, how do you fancy
> taking a stab at a patch? We can give you some pointers to get started
> if required.
>
> Mark
>
>
>>
>> In catalina logs I see
>>
>> 08-Dec-2017 22:02:05.532 INFO [ajp-nio-8019-exec-1]
>> org.apache.catalina.core.ApplicationContext.log HTMLManager: restart:
>> Reloading web application '/p'
>> 08-Dec-2017 22:02:05.533 INFO [ajp-nio-8019-exec-1]
>> org.apache.catalina.core.StandardContext.reload Reloading Context with
>> name [/p] has started
>> 08-Dec-2017 22:02:14.596 INFO [ajp-nio-8019-exec-1]
>> org.apache.catalina.core.StandardContext.reload Reloading Context with
>> name [/p] is completed
>>
>> It seems catalina knows the value of the context path for the webapp
>> when context.xml is being digested. Should I post an RFE on BZ to have
>> it added to the system properties, or am I incorrect in this
>> assumption?
>>
>>
>>
>>>
>>>>
>>>> I tried following the source through for DirResourceSet but couldn't
>>>> see where even catalina.base is getting replaced.
>>>>
>>>> (I know ant is a solution and I eventually need it for other things
>>>> too, but I have never used it and it's not a learning rabbit-hole I
>>>> can go down right now)
>>>>
>>
>> -
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: context resources, replacement parameters

[Chris Cheshire]

On Fri, Dec 8, 2017 at 3:36 PM, Mark Thomas <ma...@apache.org> wrote:
> On 08/12/17 18:49, Chris Cheshire wrote:
>> I have a directory resource set defined in my context.xml to handle images :
>>
>> 
>>   > base="${catalina.base}/cdn/p/images"
>> webAppMount="/images" />
>> 
>>
>> The /p in there actually represents the context path - a given sandbox
>> might have the same webapp deployed at different context paths
>> representing different development branches. Is it possible to use a
>> replacement parameter similar to catalina.base to replace the context
>> path the webapp is deployed at?
>
> Yes. You can use ant style property replacement in any XML file that is
> processed by the digester (server.xml, context.xml, web.xml)
>
> See the opening section of
>
> http://tomcat.apache.org/tomcat-9.0-doc/config/index.html
>
> for details.
>
> Mark
>

I logged the System properties and there is nothing in there for the
context path.

In catalina logs I see

08-Dec-2017 22:02:05.532 INFO [ajp-nio-8019-exec-1]
org.apache.catalina.core.ApplicationContext.log HTMLManager: restart:
Reloading web application '/p'
08-Dec-2017 22:02:05.533 INFO [ajp-nio-8019-exec-1]
org.apache.catalina.core.StandardContext.reload Reloading Context with
name [/p] has started
08-Dec-2017 22:02:14.596 INFO [ajp-nio-8019-exec-1]
org.apache.catalina.core.StandardContext.reload Reloading Context with
name [/p] is completed

It seems catalina knows the value of the context path for the webapp
when context.xml is being digested. Should I post an RFE on BZ to have
it added to the system properties, or am I incorrect in this
assumption?



>
>>
>> I tried following the source through for DirResourceSet but couldn't
>> see where even catalina.base is getting replaced.
>>
>> (I know ant is a solution and I eventually need it for other things
>> too, but I have never used it and it's not a learning rabbit-hole I
>> can go down right now)
>>

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 8.5 - multiple host configuration question

[Chris Cheshire]

On Fri, Dec 8, 2017 at 11:25 AM, Christopher Schultz
<ch...@christopherschultz.net> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris,
>
> On 12/7/17 2:08 PM, Chris Cheshire wrote:
>> On Thu, Sep 7, 2017 at 5:30 PM, Christopher Schultz
>> <ch...@christopherschultz.net> wrote:
>>>>
>>>> What should the permissions, owner & group be set to for
>>>> CATALINA_HOME if I am running separate instances per user?
>>>
>>> It doesn't really matter. You just need to make sure that your
>>> "users" can read the default config files -- especially
>>> conf/web.xml and conf/tomcat.xml which usually shouldn't be
>>> modified from their defaults anyway.
>>>
>>> I've always been irritated that the conf/ directory is only
>>> readable by the owner in the tarball. Maybe I'll agitate to get
>>> that changed, and only protect conf/server.xml and
>>> conf/tomcat-users.xml in that way.
>>>
>>
>> Resurrecting this 
>>
>> I'm doing some cleanup and upgrading to 8.5.24. Previously I had
>> copied the entire conf directory from HOME to BASE, and modifying
>> files as necessary. Now I removed from BASE files I hadn't touched
>> (web.xml, jaspic stuff etc), but subsequently get the following
>> message in catalina.out
>>
>> INFO ...
>> org.apache.catalina.startup.ContextConfig.getDefaultWebXmlFragment
>> No global web.xml found
>>
>> All other startup succeeds but nothing is accessible, I just get a
>> standard 404 when trying to access my web apps or even the manager
>> app. There are no actual ERROR level messages though.
>>
>> Permissions are as follows :
>>
>> /usr/local/apache-tomcat-8.5.24/conf [root@s3 conf]# ls -al total
>> 236 drwxr-x--- 2 root tomcat   4096 Nov 27 13:33 . drwxr-xr-x 9
>> root root 4096 Dec  7 16:30 .. -rw-r- 1 root tomcat  13824
>> Nov 27 13:33 catalina.policy -rw-r- 1 root tomcat   7376 Nov 27
>> 13:33 catalina.properties -rw-r- 1 root tomcat   1338 Nov 27
>> 13:33 context.xml -rw-r- 1 root tomcat   1149 Nov 27 13:33
>> jaspic-providers.xml -rw-r- 1 root tomcat   2313 Nov 27 13:33
>> jaspic-providers.xsd -rw-r- 1 root tomcat   3622 Nov 27 13:33
>> logging.properties -rw--- 1 root tomcat   7511 Nov 27 13:33
>> server.xml -rw--- 1 root tomcat   2164 Nov 27 13:33
>> tomcat-users.xml -rw-r- 1 root tomcat   2633 Nov 27 13:33
>> tomcat-users.xsd -rw-r- 1 root tomcat 169322 Nov 27 13:33
>> web.xml
>>
>> /home/sandbox1/tomcat/conf [sandbox1@s3 conf]$ ls -la total 32
>> drwxr-xr-x  3 sandbox1 sandbox1 4096 Dec  7 19:01 . drwxr-xr-x 10
>> sandbox1 sandbox1 4096 Dec  7 18:59 .. drwxr-xr-x  3 sandbox1
>> sandbox1 4096 Sep  7 16:50 Catalina -rw-r--r--  1 sandbox1 sandbox1
>> 7407 Nov  2 01:58 catalina.properties -rw-r--r--  1 sandbox1
>> sandbox1 1437 Sep  7 20:38 context.xml -rw-r--r--  1 sandbox1
>> sandbox1 3770 Dec  7 18:46 logging.properties -rw-r--r--  1
>> sandbox1 sandbox1 2522 Sep  7 20:29 server.xml
>>
>> My sandbox users belong to the 'tomcat' group (not using a
>> 'tomcat' user though). I can cat web.xml with a sandbox user. (I
>> tweaked the permissions from the defaults to allow sandbox users to
>> read the default config)
>>
>> If I copy web.xml from HOME/conf to BASE/conf everything works
>> again. So do I need to copy everything over from HOME/conf to
>> BASE/conf even if I am not changing anything?
>
> I checked, and my CATALINA_BASE/conf contains the following:
>
> server.xml (required)
> Catalina/ (and friends, optional)
> tomcat-users.xml (optional)
> web.xml (evidently required)
>
> We should probably allow web.xml to come from
> CATALINA_HOME/conf/web.xml if it's not present in CATALINA_BASE/conf/.
> I would have expected that to be allowed, but I guess it isn't.
>
> Can you file a BZ enhancement request?
>
> - -chris

Done. https://bz.apache.org/bugzilla/show_bug.cgi?id=61877

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

context resources, replacement parameters

[Chris Cheshire]

I have a directory resource set defined in my context.xml to handle images :


  


The /p in there actually represents the context path - a given sandbox
might have the same webapp deployed at different context paths
representing different development branches. Is it possible to use a
replacement parameter similar to catalina.base to replace the context
path the webapp is deployed at?

I tried following the source through for DirResourceSet but couldn't
see where even catalina.base is getting replaced.

(I know ant is a solution and I eventually need it for other things
too, but I have never used it and it's not a learning rabbit-hole I
can go down right now)

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: daemon.sh and multiple tomcat instances

[Chris Cheshire]

On Fri, Dec 8, 2017 at 11:33 AM, Christopher Schultz
<ch...@christopherschultz.net> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris,
>
> On 12/7/17 3:47 PM, Chris Cheshire wrote:
>> Apologies in advance for semi-OT sysadmin question.
>>
>> Tomcat 8.5.24, running on Centos 6. I have built jsvc according to
>> instructions in distribution. I have two tomcat instances under
>> different users (sandbox1, sandbox2) that have their own
>> CATALINA_BASE. I can start these manually from the command line
>> without a problem.
>>
>> How do I set up daemon.sh to run on boot for multiple users? Do I
>> need to (as root), make a copy of daemon.sh under /etc/init.d for
>> each user (tc_sandbox1, tc_sandbox2) and edit those to set the
>> environment variables in it (CATALINA_HOME, CATALINA_BASE,
>> JAVA_HOME, TOMCAT_USER), then use chkconfig to install the
>> scripts?
>>
>> This seems kind of clunky to set up. Is this the right way to go
>> about it or am I making it more complicated than it needs to be?
>
> Don't duplicate daemon.sh. Instead, build a wrapper script that looks
> like this:
>
> #!/bin/sh
>
> export JAVA_HOME=... [if necessary]
> export TOMCAT_USER=...
> export CATALINA_HOME=...
> export CATALINA_BASE=...
>
> $CATALINA_HOME/bin/daemon.sh
>
> That will ensure you are always up-to-date with the latest daemon.sh
> that exists in your CATALINA_BASE. It also makes your scripts
> super-easy to write and maintain.
>
> If you are using a systemd-based OS, I think there are better options
> available, but I'm not really sure.
>
 - -chris

So yes, I was making it more complicated than it needed to be. Cheers :)

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

daemon.sh and multiple tomcat instances

[Chris Cheshire]

Apologies in advance for semi-OT sysadmin question.

Tomcat 8.5.24, running on Centos 6. I have built jsvc according to
instructions in distribution. I have two tomcat instances under
different users (sandbox1, sandbox2) that have their own
CATALINA_BASE. I can start these manually from the command line
without a problem.

How do I set up daemon.sh to run on boot for multiple users? Do I need
to (as root), make a copy of daemon.sh under /etc/init.d for each user
(tc_sandbox1, tc_sandbox2) and edit those to set the environment
variables in it (CATALINA_HOME, CATALINA_BASE, JAVA_HOME,
TOMCAT_USER), then use chkconfig to install the scripts?

This seems kind of clunky to set up. Is this the right way to go about
it or am I making it more complicated than it needs to be?

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 8.5 - multiple host configuration question

[Chris Cheshire]

On Thu, Sep 7, 2017 at 5:30 PM, Christopher Schultz
 wrote:
>>
>> What should the permissions, owner & group be set to for
>> CATALINA_HOME if I am running separate instances per user?
>
> It doesn't really matter. You just need to make sure that your "users"
> can read the default config files -- especially conf/web.xml and
> conf/tomcat.xml which usually shouldn't be modified from their
> defaults anyway.
>
> I've always been irritated that the conf/ directory is only readable
> by the owner in the tarball. Maybe I'll agitate to get that changed,
> and only protect conf/server.xml and conf/tomcat-users.xml in that way.
>

Resurrecting this 

I'm doing some cleanup and upgrading to 8.5.24. Previously I had
copied the entire conf directory from HOME to BASE, and modifying
files as necessary. Now I removed from BASE files I hadn't touched
(web.xml, jaspic stuff etc), but subsequently get the following
message in catalina.out

INFO ... org.apache.catalina.startup.ContextConfig.getDefaultWebXmlFragment
No global web.xml found

All other startup succeeds but nothing is accessible, I just get a
standard 404 when trying to access my web apps or even the manager
app. There are no actual ERROR level messages though.

Permissions are as follows :

/usr/local/apache-tomcat-8.5.24/conf
[root@s3 conf]# ls -al
total 236
drwxr-x--- 2 root tomcat   4096 Nov 27 13:33 .
drwxr-xr-x 9 root root 4096 Dec  7 16:30 ..
-rw-r- 1 root tomcat  13824 Nov 27 13:33 catalina.policy
-rw-r- 1 root tomcat   7376 Nov 27 13:33 catalina.properties
-rw-r- 1 root tomcat   1338 Nov 27 13:33 context.xml
-rw-r- 1 root tomcat   1149 Nov 27 13:33 jaspic-providers.xml
-rw-r- 1 root tomcat   2313 Nov 27 13:33 jaspic-providers.xsd
-rw-r- 1 root tomcat   3622 Nov 27 13:33 logging.properties
-rw--- 1 root tomcat   7511 Nov 27 13:33 server.xml
-rw--- 1 root tomcat   2164 Nov 27 13:33 tomcat-users.xml
-rw-r- 1 root tomcat   2633 Nov 27 13:33 tomcat-users.xsd
-rw-r- 1 root tomcat 169322 Nov 27 13:33 web.xml

/home/sandbox1/tomcat/conf
[sandbox1@s3 conf]$ ls -la
total 32
drwxr-xr-x  3 sandbox1 sandbox1 4096 Dec  7 19:01 .
drwxr-xr-x 10 sandbox1 sandbox1 4096 Dec  7 18:59 ..
drwxr-xr-x  3 sandbox1 sandbox1 4096 Sep  7 16:50 Catalina
-rw-r--r--  1 sandbox1 sandbox1 7407 Nov  2 01:58 catalina.properties
-rw-r--r--  1 sandbox1 sandbox1 1437 Sep  7 20:38 context.xml
-rw-r--r--  1 sandbox1 sandbox1 3770 Dec  7 18:46 logging.properties
-rw-r--r--  1 sandbox1 sandbox1 2522 Sep  7 20:29 server.xml

My sandbox users belong to the 'tomcat' group (not using a 'tomcat'
user though). I can cat web.xml with a sandbox user. (I tweaked the
permissions from the defaults to allow sandbox users to read the
default config)

If I copy web.xml from HOME/conf to BASE/conf everything works again.
So do I need to copy everything over from HOME/conf to BASE/conf even
if I am not changing anything?

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [ANN] Apache Tomcat 9.0.2 available

[Chris Cheshire]

Interesting take on 'beta', in all my years writing software I haven't
seen that used before.

I ask because I saw it has recently gone from milestones, to release
candidates to beta with different point versions. I didn't know if I
had missed something in between :)


On Fri, Dec 1, 2017 at 10:11 AM, Olaf Kock <tom...@olafkock.de> wrote:
>
> On 01.12.2017 15:54, Chris Cheshire wrote:
>>
>> Has 9 had an official release yet, or is it still almost there?
>>
>> On Fri, Dec 1, 2017 at 9:05 AM, Mark Thomas <ma...@apache.org> wrote:
>>>
>>> The Apache Tomcat team announces the immediate availability of Apache
>>> Tomcat 9.0.2 (beta).
>
>
> 9.0.2 (beta)
>
> beta = before estimated time (of) arrival
>
> I guess this is an official beta release. Does that count? ;)
>
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

SSL connectors

[Chris Cheshire]

I see in the changelog for 8.5.24

60762: Add the ability to make changes to the TLS configuration of a
connector at runtime without having to restart the Connector. (markt)

Does this mean we can now update SSL certificates without bouncing the
connector?

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [ANN] Apache Tomcat 9.0.2 available

[Chris Cheshire]

Has 9 had an official release yet, or is it still almost there?

On Fri, Dec 1, 2017 at 9:05 AM, Mark Thomas  wrote:
> The Apache Tomcat team announces the immediate availability of Apache
> Tomcat 9.0.2 (beta).
>
> Apache Tomcat 9 is an open source software implementation of the Java
> Servlet, JavaServer Pages, Java Unified Expression Language, Java
> WebSocket and JASPIC technologies.
>
> Apache Tomcat 9.0.2 is a bugfix release. The notable changes compared to
> 9.0.1 include:
>
> - Java 9 is fully supported
>
> - Fixed a number of HTTP/2 issues
>
> - Fixed numerous JASPIC issues with patches from Lazar
>
> - Update the packaged version of the Tomcat Native Library to
>   1.2.16 to pick up the latest Windows binaries built with
>   APR 1.6.3 and OpenSSL 1.0.2m
>
>
> Please refer to the change log for the complete list of changes:
> http://tomcat.apache.org/tomcat-9.0-doc/changelog.html
>
> Downloads:
> http://tomcat.apache.org/download-90.cgi
>
> Migration guides from Apache Tomcat 7.x and 8.x:
> http://tomcat.apache.org/migration.html
>
> Enjoy!
>
> - The Apache Tomcat team
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: stopping scanning of TLDs

[Chris Cheshire]

On Wed, Nov 8, 2017 at 11:24 AM, Ray Holme  wrote:
> In a prior post, I asked if there was a way to see if a jar uses  a tag 
> library.
>   Chris responded - look for ".tld" in the files.
> So I looked (turns out ecj.. has no ".tld":
> $ grep "tld" *.jarBinary file catalina-storeconfig.jar matches
> Binary file ecj-4.6.1.jar matches
> Binary file tomcat-util-scan.jar matches
> I am just trying to find a reliable way to see if there is TAG library.
>

Ray, instead of doing a grep on the jar contents as a whole, do it
based upon the file names within the jar and match the files ending in
'.tld'

$ jar tf tomcat-util-scan.jar | grep "\.tld$"
$ jar tf catalina-storeconfig.jar | grep "\.tld$"
$ jar tf ecj-4.6.3.jar | grep "\.tld$"
$ jar tf javax.servlet.jsp.jstl-1.2.1.jar | grep "\.tld$"
META-INF/fmt-1_0-rt.tld
META-INF/scriptfree.tld
META-INF/fmt-1_0.tld
META-INF/x-1_0.tld
META-INF/sql.tld
META-INF/c.tld
META-INF/x-1_0-rt.tld
META-INF/sql-1_0.tld
META-INF/sql-1_0-rt.tld
META-INF/permittedTaglibs.tld
META-INF/x.tld
META-INF/c-1_0-rt.tld
META-INF/c-1_0.tld
META-INF/fn.tld
META-INF/fmt.tld


Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: configuring jarscanner in 8.5 to ignore entire directory

[Chris Cheshire]

PS JarScannerFilter above is a copypasta error, I do have the correct
tag in my config with the same result

On Tue, Oct 31, 2017 at 3:03 PM, Chris Cheshire <yahoono...@gmail.com> wrote:
> How do I tell the jar scanner to ignore my entire WEB-INF/lib
> directory for the pluggability scan and tld scan?
>
> I have in my context.xml
>
> 
>pluggabilitySkip="${tomcat.util.scan.StandardJarScanFilter.jarsToSkip},*/WEB-INF/lib/*.jar"
>
> tldSkip="${tomcat.util.scan.StandardJarScanFilter.jarsToSkip},*/WEB-INF/lib/*.jar"
> />
> 
>
>
> the JarScanner completely ignores it as evidenced by the 20+ log
> messages along the lines of
>
> org.apache.jasper.servlet.TldScanner$TldScannerCallback.scan No TLD
> files were found in
> [file:/home/sandbox1/tomcat/webapps/p/WEB-INF/lib/java-image-scaling-0.8.6.jar].
> Consider adding the JAR to the
> tomcat.util.scan.StandardJarScanFilter.jarsToSkip property in
> CATALINA_BASE/conf/catalina.properties file
>
>
> If I start listing each individual jar file it stops scanning them. Is
> there no way to just exclude an entire directory?

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

configuring jarscanner in 8.5 to ignore entire directory

[Chris Cheshire]

How do I tell the jar scanner to ignore my entire WEB-INF/lib
directory for the pluggability scan and tld scan?

I have in my context.xml


  



the JarScanner completely ignores it as evidenced by the 20+ log
messages along the lines of

org.apache.jasper.servlet.TldScanner$TldScannerCallback.scan No TLD
files were found in
[file:/home/sandbox1/tomcat/webapps/p/WEB-INF/lib/java-image-scaling-0.8.6.jar].
Consider adding the JAR to the
tomcat.util.scan.StandardJarScanFilter.jarsToSkip property in
CATALINA_BASE/conf/catalina.properties file


If I start listing each individual jar file it stops scanning them. Is
there no way to just exclude an entire directory?

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [OT] classloader issue with bouncycastle

[Chris Cheshire]

On Thu, Oct 26, 2017 at 9:42 AM, Christopher Schultz
 wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris,
>
> Just curious... why are you using BC and not the JVM-provided crypto
> provider? What JVM are you using?
>

When I first starting looking for examples on doing PGP encryption in
Java, all I found
were (albeit obsolete) guides to doing it with BC.

JVM is OpenJDK 1.8. I first started fiddling with this using 1.6.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: tomcat 8.5.23 dbcp not honoring autocommit = false?

[Chris Cheshire]

On Thu, Oct 26, 2017 at 3:00 PM, Christopher Schultz
<ch...@christopherschultz.net> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris,
>
> On 10/16/17 9:43 AM, Chris Cheshire wrote:
>> On Fri, Oct 13, 2017 at 5:00 PM, Christopher Schultz
>> <ch...@christopherschultz.net> wrote:
>>> -BEGIN PGP SIGNED MESSAGE-
>>
>>> When you say you have "autocommit disabled in mysql config" what
>>> do you mean?
>>>
>>
>> /etc/my.cnf : [mysqld] autocommit=0
>>
>> This turns off autocommit off as a default for all connections.
>
> It only affects connections from MySQL's "mysql" command-line client.
> It does not affect e.g. Java-based clients.
>

It's in the [mysqld] section of my.cnf so it is supposed to be at a
server level.

I do have stuff in the [mysql] section that affects the command line client
behaviour only.

>> I need this at a minimum for the mysql client, but in the absence
>> of any other configuration it should be the default for a
>> connection from any client.
>
> The JDBC spec says that all connections are auto-commit unless
> otherwise specified. So if you are creating your own connections or
> using e.g. a connection-pool then you'll have to make sure that you
> configure them to be NOT auto-commit. This is not a setting that you
> can control from the server.
>
> More below.
>

I was doing that anyway in both the pool configuration and when
the connection is grabbed from the pool. I've since changed the code to not
touch the autocommit setting and leave that up to the datasource
(resource config in context.xml)


>>> On 10/13/17 10:17 AM, Chris Cheshire wrote:
>>>> 
>>>>
>>>> As a further test I just took out my explicit rollback in my
>>>> DAOFactory close() method, and swapped back to commons dbcp.
>>>> Added an update that wasn't explicitly committed, and it
>>>> correctly did not get committed when the connection was closed.
>>>> Swapped back to tomcat dbcp and repeated, it got committed
>>>> without an explicit commit statement.
>>>>
>>>> I'm really puzzled as to why *I* have to explicitly rollback
>>>> on close if autocommit is not enabled, instead of tomcat dbcp
>>>> handling that when commons dbcp appears to do it.
>>>
>>> No connection pool can read your mind. If you begin a transaction
>>> (or never start one), you must either commit or rollback. Merely
>>> calling close() does not explicitly cause either of those to be
>>> called.
>>>
>>
>> And that's just it. If I don't explicitly commit, then why are
>> changes being committed when the connection is closed and returned
>> back to the pool?
>>
>>>> If I do
>>>>
>>>> daoFactory = new MySQLDAOFactoryImpl(getDataSource());
>>>>
>>>> // update #1 daoFactory.commit()
>>>>
>>>> // update #2 daoFactory.close();
>>>>
>>>> then update #2 is being committed.
>>>
>>> I'm curious why you are doing "update #2" without either COMMIT
>>> or ROLLBACK. That seems like ... a mistake.
>>>
>>
>> Correct. This is an example to illustrate a mistake I found in my
>> code. I found a servlet that actually wasn't explicitly committing
>> when it should have been, yet everything it was doing was being
>> committed to the database.
>>
>>> - From the Connection.close() javadoc:
>>>
>>> " It is strongly recommended that an application explicitly
>>> commits or rolls back an active transaction prior to calling the
>>> close method. If the close method is called and there is an
>>> active transaction, the results are implementation-defined. "
>>>
>>
>> If a commit is not being explicitly issued, then the commit
>> behaviour should honor that of the connection, yes?
>
> Yes, but it's more complicated than that. Any change to the
> connection's settings (which happen ALL THE TIME when the connection
> is being returned to a connection pool) will cause an implicit COMMIT.
> That's why it's super important for you to either COMMIT or ROLLBACK
> yourself.
>
> Note that "autocommit = false" doesn't mean "autorollback=true".
> Best-case scenario for you there is that the transaction gets
> committed *later* when another piece of your code grabs a connection
> from the pool, does its work (successfully) and issues a COMMIT.
>
> It's just NOT the pool's job nor the driver's job to clean-up after
> an

Re: classloader issue with bouncycastle

[Chris Cheshire]

On Wed, Oct 18, 2017 at 11:31 AM, Konstantin Kolinko
<knst.koli...@gmail.com> wrote:
> 2017-10-18 17:56 GMT+03:00 Chris Cheshire <yahoono...@gmail.com>:
>> Using bouncy castle v1.58, Tomcat 8.5, java 1.8.
>>
>> I have the unlimited security policy files installed, the BC jars in
>> my WEB-INF/lib directory and in order to register the BC provider, I
>> do
>>
>> static {
>> Security.addProvider(new BouncyCastleProvider());
>> }
>>
>> in a utility class that handles the keyring
>> setup/encryption/decryption methods for me. This works great until I
>> update the jar that contains my utility class and reload the webapp.
>> Then I get an exception thrown from it being unable to locate the BC
>> provider.
>>
>> mypackage.crypto.CryptoException:
>> org.bouncycastle.openpgp.PGPException: exception on setup:
>> java.security.NoSuchAlgorithmException: class configured for
>> MessageDigest (provider: BC) cannot be found.
>> [...]
>> Caused by: java.lang.ClassNotFoundException: Illegal access: this web
>> application instance has been stopped already.
>
>
>> If I move the call
>>
>> Security.addProvider(new BouncyCastleProvider())
>>
>> into the contextInitialized() method of a ServletContextListener,
>> everything works on reloading a webapp, no matter what classes or jars
>> I update.
>>
>> Can someone explain why the static initializer breaks down here please?
>
>
> The "Illegal access: this web application instance has been stopped
> already." exception is the expected behaviour.
>
> Please see "Memory Leaks" presentation (2010) by markt here:
> http://tomcat.apache.org/presentations.html
>
> and
> http://tomcat.apache.org/tomcat-8.5-doc/class-loader-howto.html
>

Ahhh this explains it somewhat. I was wondering why I was seeing
"application already stopped" errors when the application was clearly
still running. This error is actually referring to the classloader
instance before the reload?

>
> (Maybe moving the BC.jar and its dependencies to ${catalina.home}/lib
> is sufficient to fix your issue. A more robust solution is to move the
> initialization code as well to some listener configured in
> server.xml).
>

Assuming you mean web.xml here using a ServletContextListener? This is
what I have changed to already. Should I also be doing a
Security.removeProvider() in the contextDestroyed() method of the
listener?

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

classloader issue with bouncycastle

[Chris Cheshire]

Using bouncy castle v1.58, Tomcat 8.5, java 1.8.

I have the unlimited security policy files installed, the BC jars in
my WEB-INF/lib directory and in order to register the BC provider, I
do

static {
Security.addProvider(new BouncyCastleProvider());
}

in a utility class that handles the keyring
setup/encryption/decryption methods for me. This works great until I
update the jar that contains my utility class and reload the webapp.
Then I get an exception thrown from it being unable to locate the BC
provider.

mypackage.crypto.CryptoException:
org.bouncycastle.openpgp.PGPException: exception on setup:
java.security.NoSuchAlgorithmException: class configured for
MessageDigest (provider: BC) cannot be found.
at mypackage.crypto.PGPUtils.decrypt(PGPUtils.java:304) ~[mypackage.jar:na]
at mypackage.web.action.user.priv.settings.View.view(View.java:139)
~[classes/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[na:1.8.0_141]
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[na:1.8.0_141]
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.8.0_141]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_141]
at 
net.sourceforge.stripes.controller.DispatcherHelper$6.intercept(DispatcherHelper.java:456)
[stripes-1.6.0.jar:1.6.0]
at 
net.sourceforge.stripes.controller.ExecutionContext.proceed(ExecutionContext.java:176)
[stripes-1.6.0.jar:1.6.0]
at 
mypackage.web.interceptors.AuthenticateInterceptor.intercept(AuthenticateInterceptor.java:41)
[classes/:na]
at 
net.sourceforge.stripes.controller.ExecutionContext.proceed(ExecutionContext.java:173)
[stripes-1.6.0.jar:1.6.0]
at 
net.sourceforge.stripes.controller.BeforeAfterMethodInterceptor.intercept(BeforeAfterMethodInterceptor.java:113)
[stripes-1.6.0.jar:1.6.0]
at 
net.sourceforge.stripes.controller.ExecutionContext.proceed(ExecutionContext.java:173)
[stripes-1.6.0.jar:1.6.0]
at 
net.sourceforge.stripes.controller.ExecutionContext.wrap(ExecutionContext.java:86)
[stripes-1.6.0.jar:1.6.0]
at 
net.sourceforge.stripes.controller.DispatcherHelper.invokeEventHandler(DispatcherHelper.java:454)
[stripes-1.6.0.jar:1.6.0]
at 
net.sourceforge.stripes.controller.DispatcherServlet.invokeEventHandler(DispatcherServlet.java:278)
[stripes-1.6.0.jar:1.6.0]
at 
net.sourceforge.stripes.controller.DispatcherServlet.service(DispatcherServlet.java:160)
[stripes-1.6.0.jar:1.6.0]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
[servlet-api.jar:na]
at 
net.sourceforge.stripes.controller.DynamicMappingFilter$2.doFilter(DynamicMappingFilter.java:464)
[stripes-1.6.0.jar:1.6.0]
at 
net.sourceforge.stripes.controller.StripesFilter.doFilter(StripesFilter.java:260)
[stripes-1.6.0.jar:1.6.0]
at 
net.sourceforge.stripes.controller.DynamicMappingFilter.doFilter(DynamicMappingFilter.java:451)
[stripes-1.6.0.jar:1.6.0]
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
[catalina.jar:8.5.23]
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
[catalina.jar:8.5.23]
at 
org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)
[urlrewritefilter-4.0.3.jar:4.0.3]
at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
[urlrewritefilter-4.0.3.jar:4.0.3]
at 
org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
[urlrewritefilter-4.0.3.jar:4.0.3]
at 
org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:394)
[urlrewritefilter-4.0.3.jar:4.0.3]
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
[catalina.jar:8.5.23]
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
[catalina.jar:8.5.23]
at 
org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108)
[catalina.jar:8.5.23]
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
[catalina.jar:8.5.23]
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
[catalina.jar:8.5.23]
at 
ch.qos.logback.classic.helpers.MDCInsertingServletFilter.doFilter(MDCInsertingServletFilter.java:51)
[logback-classic-1.0.9.jar:na]
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
[catalina.jar:8.5.23]
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
[catalina.jar:8.5.23]
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
[catalina.jar:8.5.23]
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
[catalina.jar:8.5.23]
at 

Re: tomcat 8.5.23 dbcp not honoring autocommit = false?

[Chris Cheshire]

On Tue, Oct 17, 2017 at 3:44 AM, Keiichi Fujino  wrote:
> Hi
>
> You have set factory="org.apache.tomcat.jdbc.pool.DataSourceFactory".
> In other words, you do not use (tomcat)DBCP, you are using Tomcat jdbc-pool.

That's what I meant sorry. Was comparing to commons-dbcp and went
dyslexic on the acronyms.

>
> In DBCP, the default of rollbackOnReturn attribute is true.
> However, in Tomcat jdbc-pool, the default of rollbackOnReturn( and
> commitOnReturn
> ) attribute are false.
>
> see:
> http://commons.apache.org/proper/commons-dbcp/configuration.html
> http://tomcat.apache.org/tomcat-8.5-doc/jdbc-pool.html
>
>

Now that explains it entirely. Thank you so much!

Part of this is me failing to RTFM entirely, and then part is the nature of
configuration references - if you don't know what you are looking for
it is easy to miss
important details. It would be helpful if that page explained some
of the fundamental differences from a usage perspective (not just implementation
improvements) but that's another story.

Cheers,

Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: tomcat 8.5.23 dbcp not honoring autocommit = false?

[Chris Cheshire]

On Fri, Oct 13, 2017 at 5:00 PM, Christopher Schultz
<ch...@christopherschultz.net> wrote:
> -BEGIN PGP SIGNED MESSAGE-

> When you say you have "autocommit disabled in mysql config" what do
> you mean?
>

/etc/my.cnf :
[mysqld]
autocommit=0

This turns off autocommit off as a default for all connections. I need
this at a minimum
for the mysql client, but in the absence of any other configuration it
should be the
default for a connection from any client.

> On 10/13/17 10:17 AM, Chris Cheshire wrote:
>> 
>>
>> As a further test I just took out my explicit rollback in my
>> DAOFactory close() method, and swapped back to commons dbcp. Added
>> an update that wasn't explicitly committed, and it correctly did
>> not get committed when the connection was closed. Swapped back to
>> tomcat dbcp and repeated, it got committed without an explicit
>> commit statement.
>>
>> I'm really puzzled as to why *I* have to explicitly rollback on
>> close if autocommit is not enabled, instead of tomcat dbcp handling
>> that when commons dbcp appears to do it.
>
> No connection pool can read your mind. If you begin a transaction (or
> never start one), you must either commit or rollback. Merely calling
> close() does not explicitly cause either of those to be called.
>

And that's just it. If I don't explicitly commit, then why are changes being
committed when the connection is closed and returned back to the pool?

>> If I do
>>
>> daoFactory = new MySQLDAOFactoryImpl(getDataSource());
>>
>> // update #1 daoFactory.commit()
>>
>> // update #2 daoFactory.close();
>>
>> then update #2 is being committed.
>
> I'm curious why you are doing "update #2" without either COMMIT or
> ROLLBACK. That seems like ... a mistake.
>

Correct. This is an example to illustrate a mistake I found in my code. I
found a servlet that actually wasn't explicitly committing when it should
have been, yet everything it was doing was being committed to the database.

> - From the Connection.close() javadoc:
>
> "
> It is strongly recommended that an application explicitly commits or
> rolls back an active transaction prior to calling the close method. If
> the close method is called and there is an active transaction, the
> results are implementation-defined.
> "
>

If a commit is not being explicitly issued, then the commit behaviour
should honor that of the connection, yes?


> There *is* an implicit COMMIT executed if the autocommit flag is
> flipped for any reason, either true->false or false->true.
>
> If you have autocommit=false in your  configuration (which
> you do), then calling setAutoCommit(false) shouldn't do anything.
>
>> If I put in this in the close() method of my DAO Factory
>>
>> if (!this.dbConn.getAutoCommit()) { this.dbConn.rollback(); }
>>
>> before the close() call, then update #2 is correctly not getting
>> committed.
>
> This is probably the wrong approach: your close() method doesn't know
> whether it's better to call commit() or rollback(), so it should do
> neither.

I realise this too, however I have to have it in otherwise if an
exception is thrown,
then work is being committed regardless of the fact that I have autocommit
turned OFF in 3 levels, all  the way back to the mysqld configuration.

This behaviour does not happen with commons dbcp, only tomcat dbcp. There is a
difference in default behaviour between the two pools when a transaction is
not explicitly committed or rolled back when a connection is closed and
returned to the pool.

I use a try-with-resources/finally to open and close my database connections,
thus short of a JVM crash, any exceptions thrown will always close the
connections
and return them to the pool.


Thought: Perhaps the transaction marker is not being reset when a
pooled connection is being handed out to a subsequent request for a
connection, and that new servlet's work is committing the work from the first
servlet that should not have been committed.

Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: tomcat 8.5.23 dbcp not honoring autocommit = false?

[Chris Cheshire]

As a further test I just took out my explicit rollback in my
DAOFactory close() method, and swapped back to commons dbcp. Added an
update that wasn't explicitly committed, and it correctly did not get
committed when the connection was closed. Swapped back to tomcat dbcp
and repeated, it got committed without an explicit commit statement.

I'm really puzzled as to why *I* have to explicitly rollback on close
if autocommit is not enabled, instead of tomcat dbcp handling that
when commons dbcp appears to do it.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: tomcat 8.5.23 dbcp not honoring autocommit = false?

[Chris Cheshire]

On Thu, Oct 12, 2017 at 11:16 PM, Christopher Schultz
<ch...@christopherschultz.net> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris,
>
> On 10/11/17 5:21 PM, Chris Cheshire wrote:
>> Working on a migration from 7 to 8.5, and in it I am now using the
>>  tomcat dbcp, instead of apache commons dbcp.> I have found that
>> with no other changes to the db code (except the factory param for
>> the resource), it is working fine other than there is an implicit
>> commit happening when I close a connection, even with autocommit
>> turned off in mysql config, resource config AND in my code.
> Your complaint is very close to my heart, here. <3
>
> Back in 2003 or so, I posted roughly this exact question to this
> mailing list with a little less ... diplomacy, shall we say?
>
>> try { this.dbConn = this.dataSource.getConnection();
>> this.dbConn.setAutoCommit(false);
>> this.dbConn.setTransactionIsolation(Connection.TRANSACTION_READ_COMMIT
> TED);
>>
>>
> }
>> catch (SQLException ex) { throw new DAOException("unable to get
>> database connection", ex); }
>
> I'll bet you've had this problem for a really long time, but just
> didn't notice it until now.
>

Nope, only since swapping from commons dbcp (tomcat 7.x) to tomcat
dbcp in development.
I started with 8.5.20 and upgraded yesterday to 8.5.23 and it still
exhibits this behaviour.

> The core problem is that you have autocommit=false in your
> configuration and autocommit=true in your code. If an exception occurs
> and you don't rollback the transaction, the connection pool will reset
> all of the settings to your configured settings (including
> autocommit=true). Setting autocommit=true when autocommit=false
> commits the transaction, which is SUPER surprising to anyone who
> hasn't read the Javadoc[1]
>

I *don't* have autocommit=true in code, unless

this.dbConn.setAutoCommit(false);

doesn't mean what I think it means. You even have it in your example!


> Technically, this happens whether you encounter an exception or not,
> but it's fairly rare to have code that intentionally does this:
>
> conn.setAutoCommit(false);
> // UPDATE ...;
> conn.close();
>
> So, given that this is usually an "exceptional" situation, it's your
> exceptions you need to carefully handle. In fact, you need to do more
> than you are used to doing.
>
> Have a look at this post I did years later when related questions kept
> coming up on the list:
> http://blog.christopherschultz.net/index.php/2009/03/16/properly-handlin
> g-pooled-jdbc-connections/
>

I have autocommit set to false in 3 ways :

1) /etc/my.conf : autocommit=0
2) context.xml  resource def : defaultAutoCommit=false
3) in code : dbConn.setAutoCommit(false)

When I query autocommit on the connection it returns false, yet
transactions are being committed when I issue a close() on the
connection after making changes and not explicitly committing.

Color me very, very confused.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

tomcat 8.5.23 dbcp not honoring autocommit = false?

[Chris Cheshire]

Working on a migration from 7 to 8.5, and in it I am now using the
tomcat dbcp, instead of apache commons dbcp. I have found that with no
other changes to the db code (except the factory param for the
resource), it is working fine other than there is an implicit commit
happening when I close a connection, even with autocommit turned off
in mysql config, resource config AND in my code.

Resource config :



Only thing changed in that from 7.x to 8.5.x is the factory was
org.apache.commons.dbcp.BasicDataSourceFactory. I am using Connector/J
5.1.44 (latest version).


Getting a connection boils down to this in my code (pieces pulled out
of factories and other classes)

(Support class in web code)
public static DataSource getDataSource() {
try {
return (DataSource)new
InitialContext().lookup("java:comp/env/" +
ServletContextParameters.getDatabaseResourceName());
}
catch (NamingException ex) {
throw new RuntimeException("unable to find datasource", ex);
}
}


(DAO Factory implementation)
public MySQLDAOFactoryImpl(@NotNull DataSource dataSource) {
this.dataSource = dataSource;

try {
this.dbConn = this.dataSource.getConnection();
this.dbConn.setAutoCommit(false);

this.dbConn.setTransactionIsolation(Connection.TRANSACTION_READ_COMMITTED);
}
catch (SQLException ex) {
throw new DAOException("unable to get database connection", ex);
}
}

@Override
public void close() {
try {
if (this.dbConn != null) {
this.dbConn.close();
}
}
catch (SQLException ex) {
throw new DAOException("error closing database connection", ex);
}
}


If I do

daoFactory = new MySQLDAOFactoryImpl(getDataSource());

// update #1
daoFactory.commit()

// update #2
daoFactory.close();

then update #2 is being committed.

If I put in this in the close() method of my DAO Factory

if (!this.dbConn.getAutoCommit()) {
this.dbConn.rollback();
}

before the close() call, then update #2 is correctly not getting committed.

I looked back through the recent tomcat changelogs, and found a
reference to https://bz.apache.org/bugzilla/show_bug.cgi?id=61425
under the 8.5.21 release which looks like it might be addressing this
problem. However, when I download the source for 8.5.23, there is no
org.apache.tomcat.jdbc directory so I can't dive in there.

Anyone else experienced this? Hopefully I am just missing something obvious.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: mod_alias equivalent for tomcat 8?

[Chris Cheshire]

On Mon, Sep 11, 2017 at 9:40 AM, Mark Thomas <ma...@apache.org> wrote:
> On 08/09/17 23:00, Chris Cheshire wrote:
>> In tomcat8, how do I go about configuring a Resources element to
>> replace mod_alias directives in httpd?
>
> 
>
>> I have tried all kinds of variations of
>>
>>   
>> >   className="org.apache.catalina.webresources.DirResourceSet"
>>   webAppPath="/images" />
>>   
>>
>>
>> in my context.xml and I get nothing but 404s from tomcat.
>
> Check the logs for error messages. You need "webAppMount" rather than
> "webAppPath". The following works for me:
>

/facepalm

There was nothing however in catalina.log. Fiddling around, it seems that is due
to the way apps are deployed on the fly via the manager webapp vs
tomcat restarting.

I usually deploy as an exploded directory due to needing to change configuration
before deployment (no, I don't have an ant build that sets it all up,
lack of time/resources)
and on the fly changes of JS/CSS/JSPs. (I know that isn't a good practice for a
production deployment, but it is what I have to work with)

I found ROOT.xml in conf/Catalina/${hostname} as a copy of META-INF/context.xml
BEFORE I made the changes, therefore the invalid config was never being read,
hence no errors.

I was under the impression that restarting tomcat picked up changes to
META-INF/context.xml. However it seems that if the app is deployed on the
fly with the manager webapp it copies context.xml into the conf tree,
but if the exploded
directory is present when tomcat restarts it doesn't copy the
context.xml into the conf tree.
Why doesn't tomcat copy context.xml into the conf tree if it doesn't
exist when it
deploys a webapp on startup?

Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 8.5 - multiple host configuration question

[Chris Cheshire]

On Thu, Sep 7, 2017 at 5:29 PM, Christopher Schultz
<ch...@christopherschultz.net> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris,
>
> On 9/5/17 3:39 PM, Chris Cheshire wrote:
>> On Tue, Sep 5, 2017 at 2:07 PM, Christopher Schultz
>>> If I were king, I'd set things up like this:
>>>
>>> 1. Tomcat is installed in /usr/local/tomcat (or
>>> /usr/local/tomcat-x.y.z, or /opt/whatever, etc.). 2. Tomcat is
>>> never launched with CATALINA_BASE=/usr/local/tomcat 3. Each user
>>> has their own CATALINA_BASE directory in their own home directory
>>> (or wherever in the fs tree). No need to put anything in
>>> /usr/local which is usually considered to be shared and
>>> read-only. CATALINA_BASE is just a directory with the following
>>> directories in it: work/ logs/ conf/ lib/ webapps/. Anything in
>>> there overrides anything in the CATALINA_HOME where Tomcat is
>>> installed. I'd recommend using a custom conf/server.xml and
>>> leaving everything else pretty much alone except maybe a JDBC
>>> driver in CATALINA_BASE/lib that isn't necessary for all the
>>> other Tomcats that will be running on the server.
>>>
>>> This gives you a LOT of flexibility:
>>>
>>> 1. Users run their own JVMs as their own users. Filesystem
>>> permissions become simpler. Applications require less trust (e.g.
>>> apps are running at "cschultz" instead of "tomcat7"). 2. Users
>>> can select which version of Tomcat they want to use. Just change
>>> CATALINA_BASE and restart. (Roughly speaking. If you switch major
>>> versions, you'll likely have to update
>>> CATALINA_BASE/conf/server.xml quite a bit). No more "we are all
>>> running x.y.z whether you like it or not".
>>
>>
>> Ok this helps a bit for upgrades. I would just expand the new
>> tarball in a similar place, update user level conf and restart each
>> instance when ready?
>
> Exactly. Your users can even decide when they want to switch to a new
> Tomcat version.
>
>>> 3. Users can start/stop their own Tomcat services. No more
>>> emailing an administrator and asking for a restart, and having to
>>> coordinate it with several other unrelated teams who weren't
>>> expecting a service restart in the middle of the day. 4. You
>>> (admin) don't have to babysit everyone's web applications. Users
>>> simply put their own apps in CATALINA_BASE/webapps and move on
>>> with their lives.
>>>
>>
>> This means I need to configure each server and connector element
>> with different ports for each user, correct?
>
> Yes. A regimented port assignment scheme is recommended. In my shared
> development environments, I assign every dev a number and their port
> numbers become:
>
> Tomcat AJP:   8[dev #][app #]5
> Tomcat shutdown:  8[dev #][app #]6
> Tomcat "Secure" port: 8[dev #][app #]7
>
> (the "secure" port is for loopback requests; we have those for certain
> applications)
>
> So for example, my primary app id is 1 and my dev id is 2:
>
> AJP:  8215
> Shutdown: 8216
> Secure:   8217
>
>> I am fronting tomcat with httpd using an ajp connector to handle
>> ssl certs. I use letsencrypt, and on a production server I can't
>> afford to bounce even the connector and lose connections. httpd
>> handles it a lot more gracefully. Can I have separate mod_jk.conf
>> and workers.properties files for mod_jk pointing to different ports
>> for separate connectors for tomcat?
>
> Absolutely. Using regimented port assignments allows you to set up
> everyone's port assignments in advance using a template worker and
> then a bunch of workers that all look the same except for the port
> numbers.
>
> Then you just need to map URLs (e.g. /dev1-app1) to the matching port
> numbers.
>
>>>> What about file/directory permissions, assuming tomcat is
>>>> running under the 'tomcat' user? I have root access to the
>>>> machine, so changing groups, users, permissions is not an
>>>> issue.
>>>
>>> Free yourself from the "tomcat user". It's one of the things I
>>> dislike most about the package-managed versions of Tomcat: they
>>> tend to run everything as a single user which is completely
>>> unnecessary.
>>>
>>
>> Does this mean I launch tomcat (CATALINA_BASE/bin/startup.sh) as
>> each user (sandbox1, sandbox2 etc)?
>
> Yes. You may see that as a Good Thing or a Bad Thing. I think it's Good.
>
>&

Re: 8.5 - multiple host configuration question

[Chris Cheshire]

On Thu, Sep 7, 2017 at 5:30 PM, Christopher Schultz
<ch...@christopherschultz.net> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris,
>
> On 9/5/17 4:42 PM, Chris Cheshire wrote:
>> On Tue, Sep 5, 2017 at 2:07 PM, Christopher Schultz
>> <ch...@christopherschultz.net> wrote:
>>> If I were king, I'd set things up like this:
>>>
>>> 1. Tomcat is installed in /usr/local/tomcat (or
>>> /usr/local/tomcat-x.y.z, or /opt/whatever, etc.).
>>
>>
>> Looks like I do need to adjust default permissions on this if I
>> expand as root.
>>
>> The tarball leaves me with
>>
>> [root@host apache-tomcat-8.5.20]# ls -al total 124 drwxr-xr-x  9
>> root root  4096 Sep  5 20:31 . drwxr-xr-x 14 root root  4096 Sep  5
>> 20:31 .. -rw-r-  1 root root 57092 Aug  2 21:36 LICENSE
>> -rw-r-  1 root root  1723 Aug  2 21:36 NOTICE -rw-r-  1
>> root root  7064 Aug  2 21:36 RELEASE-NOTES -rw-r-  1 root root
>> 15946 Aug  2 21:36 RUNNING.txt drwxr-x---  2 root root  4096 Sep  5
>> 20:31 bin drwx--  2 root root  4096 Aug  2 21:36 conf
>> drwxr-x---  2 root root  4096 Sep  5 20:31 lib drwxr-x---  2 root
>> root  4096 Aug  2 21:35 logs drwxr-x---  2 root root  4096 Sep  5
>> 20:31 temp drwxr-x---  7 root root  4096 Aug  2 21:36 webapps
>> drwxr-x---  2 root root  4096 Aug  2 21:35 work
>>
>>
>> What should the permissions, owner & group be set to for
>> CATALINA_HOME if I am running separate instances per user?
>
> It doesn't really matter. You just need to make sure that your "users"
> can read the default config files -- especially conf/web.xml and
> conf/tomcat.xml which usually shouldn't be modified from their
> defaults anyway.
>
> I've always been irritated that the conf/ directory is only readable
> by the owner in the tarball. Maybe I'll agitate to get that changed,
> and only protect conf/server.xml and conf/tomcat-users.xml in that way.
>
> - -chris

Thanks,

I'm just wary of giving everyone read permission to something that starts out
without it, especially when installed by root. The only change I made to the
default config anyway was to remove tomcat-users.xml since I have a
JDBC realm for restricting access to the manager webapp.


Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

mod_alias equivalent for tomcat 8?

[Chris Cheshire]

In tomcat8, how do I go about configuring a Resources element to
replace mod_alias directives in httpd?


Currently my mod_alias config for httpd has

AliasMatch ^/images/(.*)$ /var/cdn/sandbox1/images/$1
AliasMatch ^/(.*)/images/(.*)$ /var/cdn/sandbox1-$1/images/$2

Order allow,deny
Allow from all



inside a virtualhost.


Now mod_jk config is now just

JkMount  /* sandbox1

to send everything to tomcat (I was unmounting /images otherwise to
get mod_alias to do its thing).

I have tried all kinds of variations of

  

  


in my context.xml and I get nothing but 404s from tomcat.


What is the difference between FileResourceSet and DirResourceSet? The
documentation explains JarResourceSet and ignores the other two. Even
the javadocs don't say much that would lead to me use over the other.

I tried to configure logging by adding

org.apache.catalina.webresources.level = FINEST

to logging.properties but nothing appears in catalina.out (yes I
restarted tomcat). I am utterly clueless when it comes to juli, I've
been using log4j and logback since dinosaurs roamed the earth.

Thanks

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 8.5 - multiple host configuration question

[Chris Cheshire]

On Tue, Sep 5, 2017 at 2:07 PM, Christopher Schultz
 wrote:
> If I were king, I'd set things up like this:
>
> 1. Tomcat is installed in /usr/local/tomcat (or
> /usr/local/tomcat-x.y.z, or /opt/whatever, etc.).


Looks like I do need to adjust default permissions on this if I expand as root.

The tarball leaves me with

[root@host apache-tomcat-8.5.20]# ls -al
total 124
drwxr-xr-x  9 root root  4096 Sep  5 20:31 .
drwxr-xr-x 14 root root  4096 Sep  5 20:31 ..
-rw-r-  1 root root 57092 Aug  2 21:36 LICENSE
-rw-r-  1 root root  1723 Aug  2 21:36 NOTICE
-rw-r-  1 root root  7064 Aug  2 21:36 RELEASE-NOTES
-rw-r-  1 root root 15946 Aug  2 21:36 RUNNING.txt
drwxr-x---  2 root root  4096 Sep  5 20:31 bin
drwx--  2 root root  4096 Aug  2 21:36 conf
drwxr-x---  2 root root  4096 Sep  5 20:31 lib
drwxr-x---  2 root root  4096 Aug  2 21:35 logs
drwxr-x---  2 root root  4096 Sep  5 20:31 temp
drwxr-x---  7 root root  4096 Aug  2 21:36 webapps
drwxr-x---  2 root root  4096 Aug  2 21:35 work


What should the permissions, owner & group be set to for CATALINA_HOME
if I am running separate instances per user?

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 8.5 - multiple host configuration question

[Chris Cheshire]

On Tue, Sep 5, 2017 at 2:07 PM, Christopher Schultz
<ch...@christopherschultz.net> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris,
>
> On 9/5/17 10:54 AM, Chris Cheshire wrote:
>> I am migrating from 7 (yum repo installation) to 8.5 (direct from
>> apache) and looking to improve configuration where possible.
>>
>> Currently (on *nix) I have a machine that runs sandboxes for my
>> domain, call them sb1.dom.com and sb2.dom.com. They each have
>> their own (system) user and in tomcat's system.xml
>
> Nit: server.xml
>

Brain fart :)


>> I have a host for each :
>>
>> 
>>
>> 
>>
>> Each has access to the host-manager app via a hardlink to
>> manager.xml through
>> /usr/share/tomcat/conf/Catalina/${hostname}/manager.xml. Each user
>> belongs to the tomcat group, and has their webapps directory group
>> readable so Tomcat can deploy the apps. Each host may have multiple
>> contexts within it representing code branches. The env variables
>> have CATALINA_HOME and CATALINA_BASE pointing to
>> /usr/share/tomcat.
>>
>> Reading RUNNING.txt, it says that HOME and BASE can point to
>> different locations for a multi-user environment, which sounds like
>> what I am doing. How do I go about configuring it this way?
>
> It depends upon your goals. If you want to run a single JVM, then it
> really doesn't matter whether you have a "single" Tomcat where
> CATALINA_HOME == CATALINA_BASE. If you want to run multiple JVMs, it's
> pretty much required that you use a split configuration.
>
> I'd argue that you should always have a split configuration, because
> it allows you to upgrade/downgrade almost trivially without disturbing
> your application's (Tomcat) configuration.
>
>> Assume I put the tomcat installation in /usr/local, with a symlink
>> from /usr/local/tomcat to
>> /usr/local/tomcat/apache-tomcat-${version}
>>
>> Would it be better to put the webapps for each user under
>> /usr/local/tomcat/webapps and symlink to them from the users home
>> directory? What would the structure look like and what would I set
>> CATALINA_BASE and CATALINA_HOME to?
>
> If I were king, I'd set things up like this:
>
> 1. Tomcat is installed in /usr/local/tomcat (or
> /usr/local/tomcat-x.y.z, or /opt/whatever, etc.).
> 2. Tomcat is never launched with CATALINA_BASE=/usr/local/tomcat
> 3. Each user has their own CATALINA_BASE directory in their own home
> directory (or wherever in the fs tree). No need to put anything in
> /usr/local which is usually considered to be shared and read-only.
> CATALINA_BASE is just a directory with the following directories in
> it: work/ logs/ conf/ lib/ webapps/. Anything in there overrides
> anything in the CATALINA_HOME where Tomcat is installed. I'd recommend
> using a custom conf/server.xml and leaving everything else pretty much
> alone except maybe a JDBC driver in CATALINA_BASE/lib that isn't
> necessary for all the other Tomcats that will be running on the server.
>
> This gives you a LOT of flexibility:
>
> 1. Users run their own JVMs as their own users. Filesystem permissions
> become simpler. Applications require less trust (e.g. apps are running
> at "cschultz" instead of "tomcat7").
> 2. Users can select which version of Tomcat they want to use. Just
> change CATALINA_BASE and restart. (Roughly speaking. If you switch
> major versions, you'll likely have to update
> CATALINA_BASE/conf/server.xml quite a bit). No more "we are all
> running x.y.z whether you like it or not".


Ok this helps a bit for upgrades. I would just expand the new tarball
in a similar
place, update user level conf and restart each instance when ready?



> 3. Users can start/stop their own Tomcat services. No more emailing an
> administrator and asking for a restart, and having to coordinate it
> with several other unrelated teams who weren't expecting a service
> restart in the middle of the day.
> 4. You (admin) don't have to babysit everyone's web applications.
> Users simply put their own apps in CATALINA_BASE/webapps and move on
> with their lives.
>


This means I need to configure each server and connector element with different
ports for each user, correct?

I am fronting tomcat with httpd using an ajp connector to handle ssl
certs. I use
letsencrypt, and on a production server I can't afford to bounce even
the connector
and lose connections. httpd handles it a lot more gracefully. Can I
have separate
mod_jk.conf and workers.properties files for mod_jk pointing to
different ports for
separate connectors for tomcat?



>> What about file/directory permissions, assuming tomcat is running
>

8.5 - multiple host configuration question

[Chris Cheshire]

I am migrating from 7 (yum repo installation) to 8.5 (direct from
apache) and looking to improve configuration where possible.

Currently (on *nix) I have a machine that runs sandboxes for my
domain, call them sb1.dom.com and sb2.dom.com. They each have their
own (system) user and in tomcat's system.xml I have a host for each :






Each has access to the host-manager app via a hardlink to manager.xml
through /usr/share/tomcat/conf/Catalina/${hostname}/manager.xml. Each
user belongs to the tomcat group, and has their webapps directory
group readable so Tomcat can deploy the apps. Each host may have
multiple contexts within it representing code branches. The env
variables have CATALINA_HOME and CATALINA_BASE pointing to
/usr/share/tomcat.

Reading RUNNING.txt, it says that HOME and BASE can point to different
locations for a multi-user environment, which sounds like what I am
doing. How do I go about configuring it this way?

Assume I put the tomcat installation in /usr/local, with a symlink
from /usr/local/tomcat to /usr/local/tomcat/apache-tomcat-${version}

Would it be better to put the webapps for each user under
/usr/local/tomcat/webapps and symlink to them from the users home
directory? What would the structure look like and what would I set
CATALINA_BASE and CATALINA_HOME to?

What about file/directory permissions, assuming tomcat is running
under the 'tomcat' user? I have root access to the machine, so
changing groups, users, permissions is not an issue.

Thanks

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: letsencrypt integration?

[Chris Cheshire]

On Thu, Aug 24, 2017 at 4:29 PM, Christopher Schultz
<ch...@christopherschultz.net> wrote:
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris,
>
> On 8/24/17 4:03 PM, Chris Cheshire wrote:
> > Cheers :)
> >
> > On Thu, Aug 24, 2017 at 3:35 PM, Mark Thomas <ma...@apache.org>
> > wrote:
> >
> >> On 24/08/17 19:50, Chris Cheshire wrote:
> >>> Currently I am using httpd to handle SSL (because my certs are
> >>> generated via LE) with all content being passed off to Tomcat 7
> >>> (investigating 8.5 upgrade).
> >>>
> >>> I had a poke around on the archives and found mention of a talk
> >>> on it in
> >> a
> >>> conference in Miami.
> >>>
> >>> http://tomcat.10.x6.nabble.com/Dynamic-reloading-of-SSL-
> >> certificates-tt5059619.html#a5059673
> >>>
> >>> Did this happen? I looked in the Tomcat youtube channel and
> >>> found a
> >> handful
> >>> of videos from there, but nothing on LE. Is it something that
> >>> is still in the "we'd like to find time to do it, but don't
> >>> know who or when" phase,
> >> or
> >>> something that is being worked on for Tomcat 9?
> >>
> >> We only had video for the final day in Miami. But we have audio
> >> for the others.
> >>
> >> http://tomcat.apache.org/presentations.html
>
> There are two items here:
>
> 1. Can Tomcat be configured and scripted for LE (pretty easy)
> 2. Tomcat can (with caveats) reload the certificate store
>
> I have not made any progress on #2. The Tomcat/LE presentation in the
> above link mentions we'll be trying to implement seamless reloading,
> but it's not done, yet. The presentation shows you how to reload it in
> a potentially disruptive way (because the connector is stopped and
> re-started, killing any in-flight requests).
>
> So it's not great, but it IS possible.
>
> - -chris


Just finished listening to your audio and following the slides. Thank
you for making these available.

Tomcat 9.0 supports .pem files, correct? What about 8.5? (I am still
using 7 and working on upgrading). With this support, does this mean
we would just reference the files certbot produces without repackaging
them into a JKS?


Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: letsencrypt integration?

[Chris Cheshire]

Cheers :)

On Thu, Aug 24, 2017 at 3:35 PM, Mark Thomas <ma...@apache.org> wrote:

> On 24/08/17 19:50, Chris Cheshire wrote:
> > Currently I am using httpd to handle SSL (because my certs are generated
> > via LE) with all content being passed off to Tomcat 7 (investigating 8.5
> > upgrade).
> >
> > I had a poke around on the archives and found mention of a talk on it in
> a
> > conference in Miami.
> >
> > http://tomcat.10.x6.nabble.com/Dynamic-reloading-of-SSL-
> certificates-tt5059619.html#a5059673
> >
> > Did this happen? I looked in the Tomcat youtube channel and found a
> handful
> > of videos from there, but nothing on LE. Is it something that is still in
> > the "we'd like to find time to do it, but don't know who or when" phase,
> or
> > something that is being worked on for Tomcat 9?
>
> We only had video for the final day in Miami. But we have audio for the
> others.
>
> http://tomcat.apache.org/presentations.html
>
> Mark
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

letsencrypt integration?

[Chris Cheshire]

Currently I am using httpd to handle SSL (because my certs are generated
via LE) with all content being passed off to Tomcat 7 (investigating 8.5
upgrade).

I had a poke around on the archives and found mention of a talk on it in a
conference in Miami.

http://tomcat.10.x6.nabble.com/Dynamic-reloading-of-SSL-certificates-tt5059619.html#a5059673

Did this happen? I looked in the Tomcat youtube channel and found a handful
of videos from there, but nothing on LE. Is it something that is still in
the "we'd like to find time to do it, but don't know who or when" phase, or
something that is being worked on for Tomcat 9?

Re: tomcat 7, null tag attributes

[Chris Cheshire]

On Tue, Jun 13, 2017 at 6:06 PM, Mark Thomas <ma...@apache.org> wrote:

> On 13/06/17 22:01, Mark Thomas wrote:
> > On 13/06/17 15:27, Chris Cheshire wrote:
>
> 
>
> >> I'm bewildered at why tomcat operates this way when it comes to Numbers
> and
> >> Strings. Why is it insistent on coercion when null and zero are
> absolutely
> >> not the same value.  If this is because of autoboxing, then isn't that a
> >> bug? A long is not a Long, and when tag attributes must be objects and
> not
> >> atomic types, shouldn't they be treated accordingly?
> >
> > Tomcat behaves this way because the specification says it has to. Plenty
> > of folks disagree strongly with the specification but Tomcat has to
> > implement it. COERCE_TO_ZERO is a specification breaking configuration
> > option to workaround this particular behaviour.
> >
> > I've been digging in to this some more and it appears that
> > COERCE_TO_ZERO has been given a wider scope in later versions of Tomcat.
> > The test case you present above behaves the way you want with
> > COERCE_TO_ZERO set to false in 8.0.x and above. I need to dig into why
> > that wider scope wasn't applied to 7.0.x.
>
> Mystery solved thanks to some svn archaeology and cross-referencing to
> the relevant specs.
>
> Tomcat 7 implements EL 2.2. EL 2.2 requires coercion of "" and null to 0
> for numeric types (section 1.18.3). (As did earlier versions of the EL
> spec).
>
> Initially there weren't many complaints because the coercion rules
> weren't implemented correctly in one important case (i.e. they left
> these as null rather than coercing to zero). Then bug 43285 [1] got
> fixed and the complaints started.
>
> To address the complaints, Tomcat introduced the system property
> org.apache.el.parser.COERCE_TO_ZERO which restored the non-specification
> compliant behaviour for the one coercion case that was causing problems.
> To align with the EL 2.2. specification, the default was for coercion to
> zero to occur.
>
> The level of complaints about the coercion rules was such that a
> backwards compatible change was introduced in EL 3.0 to not coerce to
> zero. Given Java EE's normal approach that backwards compatibility
> concerns trump all others, it is a sign of the seriousness with which
> the issue was taken that an incompatible change was made.
>
> Tomcat 8, which implemented EL 3.0, switched to the new coercion rules.
> To help migration of 7.0.x applications, the role of COERCE_TO_ZERO was
> expanded to cover all instances of EL coercion. To align with EL 3.0,
> the default was changed not to coerce to zero.
>
> Which brings us to where we are today.
>
> The problem you are seeing is a spec compliant coercion that is not
> covered by the COERCE_TO_ZERO property in 7.0.x.
>
> There are several possible solutions:
>
> 1. Upgrade to 8.x (8.5.x recommended in preference to 8.0.x)
>I appreciate that an RPM is not available for Tomcat 8 but 8.0.x has
>had stable releases for three years and 8.5.x for 1 year.
>
> 2. Use one of the workarounds. All pretty ugly.
>
> 3. Lobby for the extension of scope for COERCE_TO_ZERO in 7.0.x to be
>equivalent to the scope in 8.0.x. At this stage in the 7.0.x that is
>unlikely to happen. The risk of breakage for other users is too
>great.
>
> 4. Lobby for an additional configuration option for 7.0.x that extends
>the of scope for COERCE_TO_ZERO in 7.0.x to be equivalent to the
>scope in 8.0.x. This should be doable with care (there was some
>refactoring involved in the scope change that would need careful
>back-porting). This is also dependent on the CentOS distribution
>picking up the change to 7.0.x. I don't know how likely that is.
>Given the current package is based on a version over a year old I
>suspect that the changes of this being quick are very low.
>
> None of those options look ideal. I'd probably go with 1 but my
> familiarity with Tomcat is such that I usually prefer to work with an
> ASF distribution rather than a downstream one anyway. YMMV.
>
> Mark
>
>
> [1] https://issues.apache.org/bugzilla/show_bug.cgi?id=43285
>
>
>
Mark,

Thanks for the investigation into this.

You are right in that (1) is the best option. Finding time to translate the
ASF distribution into one with appropriate run scripts and configs, along
with a mechanism for in-place updating is tough. Having RPMs for that saves
me a lot of time which is why I stuck with 7.x for so long. In the meantime
I'll figure out the cleanest ugly workaround I can. I definitely won't
lobby for any non-security fixes in a product that is 2+ generations old
and is probably approaching EOL in the not too distant future.

Cheers,

Chris

Re: tomcat 7, null tag attributes

[Chris Cheshire]

On Tue, Jun 6, 2017 at 2:29 PM, Mark Thomas <ma...@apache.org> wrote:

> On 31/05/17 23:31, Chris Cheshire wrote:
> > I am using tomcat 7 on CentOS 7 and I need to pass a null value to tag
> > attributes of type Long/Integer/Float, however it is *always* coerced to
> > zero.
> >
> > <%@attribute name="parentId" required="true" rtexprvalue="true"
> > type="java.lang.Long" %>
> >
> > Changing required to false does nothing. I tried setting the system
> > property org.apache.el.parser.COERCE_TO_ZERO to false in tomcat.conf
> > (-Dorg.apache.el.parser.COERCE_TO_ZERO=false with my other JAVA_OPTS)
> but
> > this does nothing.
>
> As expected. That system property only affects evaluation of EL
> expressions.
>
>
But isn't  evaluation of an EL expression? Why is
it treated differently than the evaluation of ${val} when it is used in the
same scope as it is declared/assigned?

For instance, this tests in a JSP :


l is null

will succeed.

The moment I pass l into a tag and try the exact same evaluation inside, it
fails because it has been coerced to 0.


>
> > How do I pass a null Long/Float/Integer as a tag attribute and have it
> kept
> > as null and not turned into an incorrect value?
>
> Use parentId="<%=null%>" rather than parentId=""
>
> Ugly, but it does the job. Scriplets aren't coerced.
>
> Mark
>


I can't use that because I'm not trying to pass the value null, rather a
variable that possibly equates to null.

Also, if I have a custom bean

public class Foo {
  private Long val;

  public Foo() { }
  public Long getVal() { return val; }
  public void setVal(Long val) { this.val = val; }
}

and I pass an instance of Foo into a tag, then val stays as null.

It seems the only solutions are to use a sentinel value that "shouldn't"
get used (cringeworthy in its own right), or to wrap everything in a custom
bean (also extremely ugly).

I'm bewildered at why tomcat operates this way when it comes to Numbers and
Strings. Why is it insistent on coercion when null and zero are absolutely
not the same value.  If this is because of autoboxing, then isn't that a
bug? A long is not a Long, and when tag attributes must be objects and not
atomic types, shouldn't they be treated accordingly?


Chris

Re: tomcat 7, null tag attributes

[Chris Cheshire]

7.0.77 (latest version in EPEL repository)

On Thu, Jun 1, 2017 at 10:27 AM, Christopher Schultz <
ch...@christopherschultz.net> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris,
>
> On 5/31/17 6:31 PM, Chris Cheshire wrote:
> > I am using tomcat 7 on CentOS 7 and I need to pass a null value to
> > tag attributes of type Long/Integer/Float, however it is *always*
> > coerced to zero.
> >
> > <%@attribute name="parentId" required="true" rtexprvalue="true"
> > type="java.lang.Long" %>
> >
> > Changing required to false does nothing. I tried setting the
> > system property org.apache.el.parser.COERCE_TO_ZERO to false in
> > tomcat.conf (-Dorg.apache.el.parser.COERCE_TO_ZERO=false with my
> > other JAVA_OPTS) but this does nothing. The value before it hits
> > the tag is null and inside the tag is 0. If I query the System
> > properties it shows it as set to false, but Tomcat is not honoring
> > it and is still coercing nulls to zero. I understand the spec says
> > to do this etc but that defeats the purpose of using an object vs
> > atomic type in the first place and is horribly shortsighted.
> >
> > Upgrading to Tomcat 8 is not a solution unfortunately as there is
> > no RPM for it.
> >
> > How do I pass a null Long/Float/Integer as a tag attribute and have
> > it kept as null and not turned into an incorrect value?
>
> What exact version of Tomcat 7 are you running?
>
> - -chris
> -BEGIN PGP SIGNATURE-
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJZMCRCAAoJEBzwKT+lPKRYveMP/RbE4suNmhmV6Yk7+OY3iiv0
> BuM6TruMa9ijRhewZJJHBE1KSjskZjNkA7Ls8+pdKUDHNExeLSbIY+k56XHT4Yvb
> Y8pnIMeFcMTYIBHUjTNmyCYJm8B0CD+B4L5hJM/dLbVLASp82JFPw3lQt0mhsrua
> AX7bpE1pRooU1DpiB2FeJhDhmKywWzq34o5QA8jyq2egnlPD2ip0P4TwpjDe7FzM
> z2szb6lH2qI/9SWEKOxfc7FKMmtpM2kCtQO8gBY0WatGLxGlMxBAXQVGmV/70dS4
> /lIyAsKfiB1HeNMhykRniKKh6miNCvVsslF4pn1wq5MLXSmYHTSV1OpFWG5yVrLe
> NZVIJMiLO9NMQLEgjqNwJZfrdd6JB67LUQwulAM7r2AHzHl3LJI6IAxY5LXC41OY
> jRqzNCJkriJkThrC/bFYfdb28iishM0wT/q+/JLi/3M9HEPPMKJH80oDFzFsfhum
> jUfUENyVwxczUS4IAmEAPuESRZgXoXrs8h1XImH/04FJfwMxIY4Owm5+zlYH2qde
> H5qxlYwUkw035dDTBr/Wi7MPh1K7fxwWnnV4qFgPGImFzRx93C5VUO3AfCm6JDsv
> obutg31VzU7dxph1o1Bx4UsR/44wcK+y/eiEKgd3RBZNtpWuApJa7Yhuj1qtShJY
> nHGeLzQPm33MGBvL62P9
> =TQH3
> -END PGP SIGNATURE-
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

tomcat 7, null tag attributes

[Chris Cheshire]

I am using tomcat 7 on CentOS 7 and I need to pass a null value to tag
attributes of type Long/Integer/Float, however it is *always* coerced to
zero.

<%@attribute name="parentId" required="true" rtexprvalue="true"
type="java.lang.Long" %>

Changing required to false does nothing. I tried setting the system
property org.apache.el.parser.COERCE_TO_ZERO to false in tomcat.conf
(-Dorg.apache.el.parser.COERCE_TO_ZERO=false with my other JAVA_OPTS) but
this does nothing. The value before it hits the tag is null and inside the
tag is 0. If I query the System properties it shows it as set to false, but
Tomcat is not honoring it and is still coercing nulls to zero. I understand
the spec says to do this etc but that defeats the purpose of using an
object vs atomic type in the first place and is horribly shortsighted.

Upgrading to Tomcat 8 is not a solution unfortunately as there is no RPM
for it.

How do I pass a null Long/Float/Integer as a tag attribute and have it kept
as null and not turned into an incorrect value?

Chris

Re: tomcat7 tldScanJar warning

[Chris Cheshire]

On Fri, Jul 12, 2013 at 9:38 AM, Christopher Schultz 
ch...@christopherschultz.net wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA256

 Chris,

 On 7/11/13 2:51 PM, Chris Cheshire wrote:
  I am working on migrating from TC6 to 7, and noticed in the log
  file the info messages about jars being needlessly scanned. Did all
  my googling and stuff and added all the jars mentioned to the
  catalina.properties file.
 
  All the warnings are gone, but it has done absolutely nothing to
  improve startup time nor page compile time. For a setup with about
  12 hosts, and at least one context each it is still taking about 75
  seconds to start up, give or take a couple of seconds every time,
  and it still takes about 10 seconds to compile a page.
 
  The last couple of replies on
  https://issues.apache.org/bugzilla/show_bug.cgi?id=51248 seem to
  imply that doing so should increase performance, but I'm not seeing
  it.
 
  Is there really any benefit to this that I am missing? It's
  cumbersome editing this and restarting tomcat if I update a web app
  and add have to add a new dependency.

 Are you using 3.0-compatible webapps? If so, try setting
 metadata-complete=true in your web-app element: this will stop
 Tomcat from scanning the JAR files for other things (not just TLDs)
 and may improve startup time.



I changed the web.xml declaration to

web-app version=3.0
 xmlns=http://java.sun.com/xml/ns/javaee;
 xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
 xsi:schemaLocation=http://java.sun.com/xml/ns/javaee
http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd;

If that is all I need to do then, yes.



 If you want to know what Tomcat is doing during those 75-seconds, take
 some thread dumps and read them.


Alright, I'll have a hunt around and find out what to do there. I'll also
change the TC logging some more and see what it spits out. All I changed
was the line for the TLD scanner

Thanks,

Chris

tomcat7 tldScanJar warning

[Chris Cheshire]

I am working on migrating from TC6 to 7, and noticed in the log file the
info messages about jars being needlessly scanned. Did all my googling and
stuff and added all the jars mentioned to the catalina.properties file.

All the warnings are gone, but it has done absolutely nothing to improve
startup time nor page compile time. For a setup with about 12 hosts, and at
least one context each it is still taking about 75 seconds to start up,
give or take a couple of seconds every time, and it still takes about 10
seconds to compile a page.

The last couple of replies on
https://issues.apache.org/bugzilla/show_bug.cgi?id=51248 seem to imply that
doing so should increase performance, but I'm not seeing it.

Is there really any benefit to this that I am missing? It's cumbersome
editing this and restarting tomcat if I update a web app and add have to
add a new dependency.


Chris

environment variables for hostname and context name

[Chris Cheshire]

I'm reconfiguring the logging for my tomcat installation (6.0.20) to
use logback (instead of log4j) for the server logging so I can split
out error messages on a per host basis. I see that catalina.home is
exposed as an environment var and usable in the logging configuration
file - what about the hostname and context name/path?

I would like to set up a config file such that I don't have to edit it
every time I add a new sandbox for a developer (host). I want the
errors split out per host instead of one big file, because it makes it
possible to tell which sandbox had a problem with a JSP page for
instance.

I have it working if I hardcode a new appender per host going to a
separate file - something like :

  root level=INFO
appender-ref ref=tomcat /
  /root

  logger 
name=org.apache.catalina.core.ContainerBase.[Catalina].[a.domain.com]
  level=INFO
  additivity=false
appender-ref ref=tomcat_www /
  /logger

and then two appenders, one for base logging from the tomcat
container, and then one for the host a.domain.com.

  appender name=tomcat_www
class=ch.qos.logback.core.rolling.RollingFileAppender
File${catalina.home}/logs/tomcat-www.log/File
RollingPolicy class=ch.qos.logback.core.rolling.TimeBasedRollingPolicy
  
FileNamePattern${catalina.home}/logs/tomcat-www.%d{-MM-dd}.log.gz/FileNamePattern
/RollingPolicy

layout class=ch.qos.logback.classic.PatternLayout
  Pattern%-25(%date{HH:mm:ss.SSS} [%thread])
%-5level%n%logger%n%msg%n/Pattern
/layout
  /appender


Now when I log an error I see something like :

17:04:01.890 [http-8080-1] ERROR
org.apache.catalina.core.ContainerBase.[Catalina].[a.domain.com].[/foo].[default]
Servlet.service() for servlet default threw exception
org.apache.jasper.JasperException: /WEB-INF/jsp/login.jsp(17,2)
Unterminated lt;core:if tag
at 
org.apache.jasper.compiler.DefaultErrorHandler.jspError(DefaultErrorHandler.java:40)
[jasper.jar:na]

[blah blah blah]

a.domain.com is the host where the web app is deployed. foo is the
context path under which the app is deployed. So if it was able to log
this information, it must be accessible somewhere, right? If I can get
access to the hostname and context name I can specify a dynamic file
name using these variables, and then only have one appender defined no
matter how many new hosts I deploy.


Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: environment variables for hostname and context name

[Chris Cheshire]

I forced that error specifically for this example. That's not the
issue. That is an example of what I am trying to log, and it
illustrates that the information I need is available at the logging
level.


On Fri, Sep 18, 2009 at 7:01 PM, Martin Gainty mgai...@hotmail.com wrote:

 org.apache.jasper.JasperException: /WEB-INF/jsp/login.jsp(17,2)

 Unterminated core:if tag

 at 
 org.apache.jasper.compiler.DefaultErrorHandler.jspError(DefaultErrorHandler.java:40)

 [jasper.jar:na]


 this says that login.jsp at line 17 column2 has an unterminated if tag
 here is a complete example that uses if component from Struts core e.g.
 s:if test=#groupStatus.odd == true odd/s:if


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: environment variables for hostname and context name

[Chris Cheshire]

Right. The tomcat directory is available through the environment
somehow, and is accessed as ${catalina.home}. From here I can get to
the tomcat logs directory. What else is exposed this way? I am hoping
that if the host and context are available in the logging message
itself, then maybe they are/can be exposed. Those values are generated
internally by tomcat when it dumps that stack trace, they weren't part
of the original logging message.

On Fri, Sep 18, 2009 at 8:30 PM, Rainer Jung rainer.j...@kippdata.de wrote:
 On 19.09.2009 02:18, Chris Cheshire wrote:
 I'm reconfiguring the logging for my tomcat installation (6.0.20) to
 use logback (instead of log4j) for the server logging so I can split
 out error messages on a per host basis. I see that catalina.home is
 exposed as an environment var and usable in the logging configuration
 file - what about the hostname and context name/path?

 I would like to set up a config file such that I don't have to edit it
 every time I add a new sandbox for a developer (host). I want the
 errors split out per host instead of one big file, because it makes it
 possible to tell which sandbox had a problem with a JSP page for
 instance.

 I have it working if I hardcode a new appender per host going to a
 separate file - something like :

   root level=INFO
     appender-ref ref=tomcat /
   /root

   logger 
 name=org.apache.catalina.core.ContainerBase.[Catalina].[a.domain.com]
           level=INFO
           additivity=false
     appender-ref ref=tomcat_www /
   /logger

 and then two appenders, one for base logging from the tomcat
 container, and then one for the host a.domain.com.

   appender name=tomcat_www
 class=ch.qos.logback.core.rolling.RollingFileAppender
     File${catalina.home}/logs/tomcat-www.log/File
     RollingPolicy 
 class=ch.qos.logback.core.rolling.TimeBasedRollingPolicy
       
 FileNamePattern${catalina.home}/logs/tomcat-www.%d{-MM-dd}.log.gz/FileNamePattern
     /RollingPolicy

     layout class=ch.qos.logback.classic.PatternLayout
       Pattern%-25(%date{HH:mm:ss.SSS} [%thread])
 %-5level%n%logger%n%msg%n/Pattern
     /layout
   /appender


 Now when I log an error I see something like :

 17:04:01.890 [http-8080-1] ERROR
 org.apache.catalina.core.ContainerBase.[Catalina].[a.domain.com].[/foo].[default]
 Servlet.service() for servlet default threw exception
 org.apache.jasper.JasperException: /WEB-INF/jsp/login.jsp(17,2)
 Unterminated lt;core:if tag
         at 
 org.apache.jasper.compiler.DefaultErrorHandler.jspError(DefaultErrorHandler.java:40)
 [jasper.jar:na]

 [blah blah blah]

 a.domain.com is the host where the web app is deployed. foo is the
 context path under which the app is deployed. So if it was able to log
 this information, it must be accessible somewhere, right? If I can get
 access to the hostname and context name I can specify a dynamic file
 name using these variables, and then only have one appender defined no
 matter how many new hosts I deploy.

 In configuration files Log4J only support two kinds of variables:

 - system properties
 - variables defined previously inside Log4J (typically in the same
 config file)

 Both ways do not have the ability to react on context information during
 logging.

 Regards,

 Rainer

 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

reconfiguring tomcat logging

[Chris Cheshire]

Whenever tomat compiles a JSP page and there is an error in it (for
instance trying to reference a property on a bean that doesn't exist),
it is throwing (and obscuring) an exception and logging it to
catalina.out and then returning whatever html it has built to the
browser, as in :

Sep 2, 2009 2:07:28 PM org.apache.catalina.core.ApplicationDispatcher
invokeSEVERE: Servlet.service() for servlet jsp threw exception
javax.el.PropertyNotFoundException: Property 'blah' not found on com.foo.bar

(and then a ginormous stack trace)

This would be good if I knew which host actually had this error. I
have one tomcat instance server with a number of hosts, each a sandbox
for a developer.

Is there any way to configure the logging per host such that these
errors that get logged to catalina.out get logged to a host specific
log file instead of catalina.out (or as well as)? I am looking at the
document at http://tomcat.apache.org/tomcat-6.0-doc/logging.html, but
I just don't see/understand what I need to do.

This is not to be confused with logging generated from within my
context - I have that configured via logback going to its own log file
successfully - it's the tomcat engine errors that I need to deal with.

Or, if this isn't possible, is there a way to configure tomcat to
generate that stack trace about the PropertyNotFoundException as the
output html when it tries to compile the JSP instead of obscuring it
and returning a partially complete html page?

Thanks

Chris

Tomcat 6.0.20
CentOS 4
Java(TM) SE Runtime Environment (build 1.6.0_16-b01)

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: processing precedence for mod_jk config?

[Chris Cheshire]

The reason why I have the mount directives so open is because I am
using Stripes with no prefix or suffix mappings on the actions
(servlets). JSPs are handled internally (every page is served via an
action), so the urls don't have a specific pattern (/servlet, /action,
.do, etc) that can be isolated for the mount directives. It could
handle it with a minimum amount of configuration and keeps urls very
clean so I went with this.

As for the differences in configuration between the two web servers -
the old one uses servlets with a specific mapping (/servlets/* 
/*.jsp) for the mount directives and is using a much older mod_jk, the
new one is open (no specific mappings) and uses the latest mod_jk. The
apache config is the same.

My concern is why I now need the unmount directives where I didn't
need them before. Is this due to the new version of mod_jk and that's
just how it behaves, or because I have a very open mapping for the
mount directives, or something else I'm just not seeing.

Given that I have a configuration that seems to be somewhat correct
and returning the right results, I'm not hugely concerned. I just like
to know why things behave the way they do so that I roughly know what
to expect if I have to change something later :)

Chris

On Tue, Aug 25, 2009 at 7:12 PM, Christopher
Schultzch...@christopherschultz.net wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 Chris,

 On 8/25/2009 9:37 PM, Chris Cheshire wrote:
 However, even with a directory deny rule in apache conf to block the
 web-inf and meta-inf directories, requests to it are still getting
 passed to tomcat.

 That's because they aren't being treated as directories in those cases.
 Try using a Location instead of a Directory and see if that works. I
 think mod_jk takes the first crack at serving files, and then allows
 Apache to continue with the rest of its possibilities. So, if your
 mod_jk mappings also map those directories, they're going to be sent to
 Tomcat.

 If I put in a JkUnMount to those directories, then apache is
 returning a forbidden error.

 Sound like that's what you want to do, anyway, right?

 JkMount  /* worker1

 What types of URLs do you actually want Tomcat to process? For instance,
 I use Struts 1.x, j_security_check-style security, and a few JSPs, so I
 only mount /*.do, /*.jsp, and /j_security_check. If you have similar
 requirements, maybe you could tighten-up your JkMount directives.

 JkUnMount /META-INF/* worker1     # without this, apache directory
 directive to return a forbidden error doesn't happen

 Right. Instead, you get a 404 from Tomcat (which isn't so bad, honestly).

   Directory /home/www/web/ROOT/META-INF
     AllowOverride none
     Order deny,allow
     Deny from all
     Satisfy all
   /Directory

 Whatever else you do, you should leave this configuration in Apache
 httpd.conf, even if it's not actually doing anything. Later, if someone
 modifies your configuration, this might provide backup protection for you.

 Try Location in addition to the Directory, but you might just need
 the JkUnMount (or more specific JkMount directives).

 Are the JkMount directives taking precedence over apache's Directory
 directives? I have another web server running mod_jk-1.2.15, tomcat
 5.5, apache 2.0.52 and I don't have this issue.

 What are the differences in configuration, then?

 - -chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

processing precedence for mod_jk config?

[Chris Cheshire]

Hi,

I have been working on a web-app that has been running under tomcat 6
alone for a while. I am now configuring Apache to do some rewrite
rules with domains and then forward on to tomcat, which is working.

However, even with a directory deny rule in apache conf to block the
web-inf and meta-inf directories, requests to it are still getting
passed to tomcat. I confirmed these were working by disabling mod_jk
and restarting, and got the apache forbidden error. When I have mod_jk
running, it passes requests to meta-inf/context.xml to tomcat which
then returns a 404 result. It's sort of the same thing but not what I
want. If I put in a JkUnMount to those directories, then apache is
returning a forbidden error.

My environment :
-
CentOS4
Tomcat 6.0.20
Apache 2.0.52 (this is what the packaging utils wanted to install,
2.2.x isn't available)
mod_jk-1.2.28

workers.properties :

worker.list=worker1
worker.worker1.type=ajp13
worker.worker1.host=localhost
worker.worker1.port=9009

mod_jk.conf :

LoadModulejk_module  modules/mod_jk.so
JkWorkersFile /etc/httpd/conf.d/workers.properties
JkShmFile /var/log/httpd/mod_jk.shm
JkLogFile /var/log/httpd/mod_jk.log
JkLogLevelinfo
JkLogStampFormat [%a %b %d %H:%M:%S %Y] 
JkMount  /* worker1
JkUnMount /*.ico worker1
JkUnMount /*.jpg worker1
JkUnMount /*.gif worker1
JkUnMount /*.png worker1
JkUnMount /*.js worker1
JkUnMount /META-INF/* worker1 # without this, apache directory
directive to return a forbidden error doesn't happen
JkUnMount /WEB-INF/* worker1   # and this
JkMountCopy all

mod_jk.conf is included via the directive in httpd.conf :

Include conf.d/*.conf

which happens right after the LoadModule directives.

And in the virtual host directive to force a forbidden error :

  Directory /home/www/web/ROOT/META-INF
AllowOverride none
Order deny,allow
Deny from all
Satisfy all
  /Directory

  Directory /home/www/web/ROOT/WEB-INF
AllowOverride none
Order deny,allow
Deny from all
Satisfy all
  /Directory

Are the JkMount directives taking precedence over apache's Directory
directives? I have another web server running mod_jk-1.2.15, tomcat
5.5, apache 2.0.52 and I don't have this issue.

Chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

virtual hosts and tomcat-users.xml

[Chris Cheshire]

I am setting up a handful of virtual hosts on a server, and I have the
manager app configured as per the docs. How do I configure the
security for the manager app such that each virtual host has its own
tomcat-users.xml though, so that the credentials of one virtual host
can't be used log on to the manager of another?

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org