Re: Integrating migration tool into Tomcat 10

[Mark Thomas]

On 10/02/2021 17:13, Christopher Schultz wrote:
> Mark,
> 
> On 2/9/21 16:12, Mark Thomas wrote:
>> The hard part
>> here is the identification of the webapp as a Java EE app. The only
>> reliable way to do this is class scanning and that is slow.
> 
> Why not look at the system id or namespace of WEB-INF/web.xml? Anyone
> using the legacy Java EE will be using web-app with a low version and a
> namespace like http://xmlns.jcp.org/xml/ns/javaee instead of
> https://jakarta.ee/xml/ns/jakartaee.

Because Jakarta EE compliant apps can still use Java EE schemas. Use of
an old schema is not a guarantee of which API is being used.

>> A final point, which probably should have been first, is is there a
>> demand for this feature? Early indications from the users list and $work
>> is that there is (going to be) a demand for this feature.
> 
> Being able to drop your old, crufty web application onto Tomcat 10 (or
> 11?) will be an absolutely killer feature.
> 
> How many times have we had someone email the users list asking for help
> upgrading their Tomcat from 4.1 -> 9.0 and something breaks and "the
> original developer is gone; I don't know Java; I've been told this is
> high-priority and just need it done by Friday"?

Indeed.

> A feature like this can really help those situations.
> 
> One possible response to this would be "don't abandon your software; get
> your vendor to re-package the application for you" or something similar.
> But some applications, while "old" don't necessarily require conversion
> because they work /just fine/.
> 
> If this feature existed today, I would *immediately* test my own
> application on it to see if it worked (and possibly consider a migration
> to Tomcat 10 in the near-term instead of dreading it). I currently have
> zero interest in downloading the conversion tool, building it, figuring
> out how to invoke it and then install the converted WAR file into a new
> version of Tomcat. It wouldn't surprise me if there are a LOT of people
> out there feeling the same way.

Ack. I wouldn't be surprised either.

Don't forget there will always be the Tomcat 9.x version which tracks
that latest Tomcat codebase but retains the Java EE 8 API. That provides
a support route that doesn't require conversion.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Integrating migration tool into Tomcat 10

[Christopher Schultz]

Mark,

On 2/9/21 16:12, Mark Thomas wrote:

The hard part
here is the identification of the webapp as a Java EE app. The only
reliable way to do this is class scanning and that is slow.


Why not look at the system id or namespace of WEB-INF/web.xml? Anyone 
using the legacy Java EE will be using web-app with a low version and a 
namespace like http://xmlns.jcp.org/xml/ns/javaee instead of 
https://jakarta.ee/xml/ns/jakartaee.



A final point, which probably should have been first, is is there a
demand for this feature? Early indications from the users list and $work
is that there is (going to be) a demand for this feature.


Being able to drop your old, crufty web application onto Tomcat 10 (or 
11?) will be an absolutely killer feature.


How many times have we had someone email the users list asking for help 
upgrading their Tomcat from 4.1 -> 9.0 and something breaks and "the 
original developer is gone; I don't know Java; I've been told this is 
high-priority and just need it done by Friday"?


A feature like this can really help those situations.

One possible response to this would be "don't abandon your software; get 
your vendor to re-package the application for you" or something similar. 
But some applications, while "old" don't necessarily require conversion 
because they work /just fine/.


If this feature existed today, I would *immediately* test my own 
application on it to see if it worked (and possibly consider a migration 
to Tomcat 10 in the near-term instead of dreading it). I currently have 
zero interest in downloading the conversion tool, building it, figuring 
out how to invoke it and then install the converted WAR file into a new 
version of Tomcat. It wouldn't surprise me if there are a LOT of people 
out there feeling the same way.


-chris

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Integrating migration tool into Tomcat 10

[Mark Thomas]

On 10/02/2021 16:29, Romain Manni-Bucau wrote:
> Le mer. 10 févr. 2021 à 17:15, Emmanuel Bourg  a écrit :
> 
>> Le 2021-02-10 16:36, Rémy Maucherat a écrit :
>>
>>> After reading everything, I'd say it's worth adding a second integrated
>>> option, and think I'm now swaying towards the runtime option. The main
>>> problem would be the detection of legacy webapps. We could simply
>>> mandate
>>> using an explicit new attribute on the Context element (and done !) so
>>> nobody pays any needless costs.
>>
>> The runtime option still incurs a cost for the legacy applications
>> because the classes and the resources have to be filtered every time the
>> application is started. Ideally the application should be converted only
>> once.
>>
> 
> Tomcat has work/ folder for that purpose. The ClassFileTransformer and
> WebResourceFilter can dump the outputs in work/ and load it from there when
> needed at restart making it almost negligible.

What you are describing is a lazy init version of the convert at
deployment. The runtime solution is starting to look a lot more complex
than the deploy time solution.

The WebResourceFilter may be non-trivial to implement. Filtering of
static resources accessed directly is relatively simple to implement -
there is code doing this for a limited subset of cases already - but
handing resources in JARs, WARS and JARS nested in WARs will be more
complex.

Another thought has occurred to me after testing the migration tool with
Jira. There are a handful of ways that applications could obtain
resources that would not be caught (and could not be caught) by the
WebResourceFilter. Migration at the time of deployment avoids these issues.

While we don;t seem to have much consensus so far, one point we do seem
to have consensus on is that the user will need to do something to
identify the application as needing conversion.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Integrating migration tool into Tomcat 10

[Romain Manni-Bucau]

Le mer. 10 févr. 2021 à 17:15, Emmanuel Bourg  a écrit :

> Le 2021-02-10 16:36, Rémy Maucherat a écrit :
>
> > After reading everything, I'd say it's worth adding a second integrated
> > option, and think I'm now swaying towards the runtime option. The main
> > problem would be the detection of legacy webapps. We could simply
> > mandate
> > using an explicit new attribute on the Context element (and done !) so
> > nobody pays any needless costs.
>
> The runtime option still incurs a cost for the legacy applications
> because the classes and the resources have to be filtered every time the
> application is started. Ideally the application should be converted only
> once.
>

Tomcat has work/ folder for that purpose. The ClassFileTransformer and
WebResourceFilter can dump the outputs in work/ and load it from there when
needed at restart making it almost negligible.


>
> Emmanuel Bourg
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>

Re: Integrating migration tool into Tomcat 10

[Emmanuel Bourg]

Le 2021-02-10 16:36, Rémy Maucherat a écrit :


After reading everything, I'd say it's worth adding a second integrated
option, and think I'm now swaying towards the runtime option. The main
problem would be the detection of legacy webapps. We could simply 
mandate

using an explicit new attribute on the Context element (and done !) so
nobody pays any needless costs.


The runtime option still incurs a cost for the legacy applications 
because the classes and the resources have to be filtered every time the 
application is started. Ideally the application should be converted only 
once.


Emmanuel Bourg

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Integrating migration tool into Tomcat 10

[Romain Manni-Bucau]

Le mer. 10 févr. 2021 à 16:37, Rémy Maucherat  a écrit :

> On Tue, Feb 9, 2021 at 10:12 PM Mark Thomas  wrote:
>
> > Hi all,
> >
> > I've been looking at the options to integrate the Java EE to Jakarta EE
> > migration functionality into Tomcat 10.
> >
> > There are essentially two ways to do this: deployment time and runtime.
> >
> > The simplest solution to implement is probably a separate webapps-legacy
> > directory (or some other name) where WARs and DIRs added to that
> > directory get converted to Jakarta EE with the new version being placed
> > in the webapps directory. There are complexities (such as handling an
> > updates of the legacy WAR) but they should be manageable.
> >
> > A more complex version of the deploy time solution has the legacy web
> > application placed in webapps, identified as a legacy webapp and then
> > replaced with the new version (several ways to do this). The hard part
> > here is the identification of the webapp as a Java EE app. The only
> > reliable way to do this is class scanning and that is slow.
> >
> > The runtime approach converts classes and static resources as they are
> > loaded. The classes are relatively easy to handle. A
> > ClassFileTransformer can be added to the WebappClassLoader to do this.
> > The static files are more of a problem. The good news is that the
> > WebResources refactoring means static file access all goes through the
> > same code but the filtering required essentially means we'd need to load
> > static files into memory - regardless of size, convert them, and then
> > update the cache with the converted version. That is likely to have a
> > performance impact.
> >
> > Because of the performance impacts of handling static resources, the
> > runtime approach also needs a way to identify applications that need
> > conversion. I don't see a reliable, performant way to do that. Which, I
> > think, leaves us with the simple deployment time approach and something
> > (filename, special directory, something else) to mark an app as needing
> > conversion.
> >
> > A final point, which probably should have been first, is is there a
> > demand for this feature? Early indications from the users list and $work
> > is that there is (going to be) a demand for this feature.
> >
> > Thoughts?
> >
>
> Overall it seems more people would like it now, for various reasons.
> However, there's zero consensus on how since everyone came up with a
> different idea.
>
> I'd like to note the tool is the appropriate solution for most cases, so we
> cover that already. The main problem with the tool is the perceived effort
> from users.
>
> After reading everything, I'd say it's worth adding a second integrated
> option, and think I'm now swaying towards the runtime option. The main
> problem would be the detection of legacy webapps. We could simply mandate
> using an explicit new attribute on the Context element (and done !) so
> nobody pays any needless costs.
>

s/attribute/listener/ ? the context listener can install the class file
transformer + webresource filter wrapper programmatically and is a one
liner for end user. Wdyt?


>
> Rémy
>
>
> >
> > Mark
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> > For additional commands, e-mail: dev-h...@tomcat.apache.org
> >
> >
>

Re: Integrating migration tool into Tomcat 10

[Rémy Maucherat]

On Tue, Feb 9, 2021 at 10:12 PM Mark Thomas  wrote:

> Hi all,
>
> I've been looking at the options to integrate the Java EE to Jakarta EE
> migration functionality into Tomcat 10.
>
> There are essentially two ways to do this: deployment time and runtime.
>
> The simplest solution to implement is probably a separate webapps-legacy
> directory (or some other name) where WARs and DIRs added to that
> directory get converted to Jakarta EE with the new version being placed
> in the webapps directory. There are complexities (such as handling an
> updates of the legacy WAR) but they should be manageable.
>
> A more complex version of the deploy time solution has the legacy web
> application placed in webapps, identified as a legacy webapp and then
> replaced with the new version (several ways to do this). The hard part
> here is the identification of the webapp as a Java EE app. The only
> reliable way to do this is class scanning and that is slow.
>
> The runtime approach converts classes and static resources as they are
> loaded. The classes are relatively easy to handle. A
> ClassFileTransformer can be added to the WebappClassLoader to do this.
> The static files are more of a problem. The good news is that the
> WebResources refactoring means static file access all goes through the
> same code but the filtering required essentially means we'd need to load
> static files into memory - regardless of size, convert them, and then
> update the cache with the converted version. That is likely to have a
> performance impact.
>
> Because of the performance impacts of handling static resources, the
> runtime approach also needs a way to identify applications that need
> conversion. I don't see a reliable, performant way to do that. Which, I
> think, leaves us with the simple deployment time approach and something
> (filename, special directory, something else) to mark an app as needing
> conversion.
>
> A final point, which probably should have been first, is is there a
> demand for this feature? Early indications from the users list and $work
> is that there is (going to be) a demand for this feature.
>
> Thoughts?
>

Overall it seems more people would like it now, for various reasons.
However, there's zero consensus on how since everyone came up with a
different idea.

I'd like to note the tool is the appropriate solution for most cases, so we
cover that already. The main problem with the tool is the perceived effort
from users.

After reading everything, I'd say it's worth adding a second integrated
option, and think I'm now swaying towards the runtime option. The main
problem would be the detection of legacy webapps. We could simply mandate
using an explicit new attribute on the Context element (and done !) so
nobody pays any needless costs.

Rémy


>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>

Re: Integrating migration tool into Tomcat 10

[Martin Grigorov]

On Wed, Feb 10, 2021 at 12:24 PM Mark Thomas  wrote:

> On 10/02/2021 09:30, Martin Grigorov wrote:
>
> 
>
> > Believe me it is not that simple.
> > Spring Boot Maven/Gradle plugins produce a jar file that contains other
> jar
> > files inside. Those jar files are not compressed for optimization reasons
> > and the Jakarta EE migration tool cannot handle that because
> > java.util.jar.* APIs does not provide a way to not compress, after
> > re-writing the classes/files.
> > So, a Spring Boot application cannot be really migrated to Jakarta EE.
>
> This is no longer correct. Use of the "-zipInMemory" option will address
> this.
>

Good to know!


>
> There are other complications with Spring Boot applications (you need to
> swap out the Tomcat implementation JARs as well) but those should be
> solvable. Whether it is the migration tool's job to solve them is a
> different question.
>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>

Re: Integrating migration tool into Tomcat 10

[Mark Thomas]

On 10/02/2021 09:30, Martin Grigorov wrote:



> Believe me it is not that simple.
> Spring Boot Maven/Gradle plugins produce a jar file that contains other jar
> files inside. Those jar files are not compressed for optimization reasons
> and the Jakarta EE migration tool cannot handle that because
> java.util.jar.* APIs does not provide a way to not compress, after
> re-writing the classes/files.
> So, a Spring Boot application cannot be really migrated to Jakarta EE.

This is no longer correct. Use of the "-zipInMemory" option will address
this.

There are other complications with Spring Boot applications (you need to
swap out the Tomcat implementation JARs as well) but those should be
solvable. Whether it is the migration tool's job to solve them is a
different question.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Integrating migration tool into Tomcat 10

[Martin Grigorov]

Hi Romain,

On Wed, Feb 10, 2021 at 10:39 AM Romain Manni-Bucau 
wrote:

> Le mer. 10 févr. 2021 à 09:32, Martin Grigorov  a
> écrit :
>
> > Hi,
> >
> > On Tue, Feb 9, 2021 at 11:12 PM Mark Thomas  wrote:
> >
> > > Hi all,
> > >
> > > I've been looking at the options to integrate the Java EE to Jakarta EE
> > > migration functionality into Tomcat 10.
> > >
> > > There are essentially two ways to do this: deployment time and runtime.
> > >
> > > The simplest solution to implement is probably a separate
> webapps-legacy
> > > directory (or some other name) where WARs and DIRs added to that
> > > directory get converted to Jakarta EE with the new version being placed
> > > in the webapps directory. There are complexities (such as handling an
> > > updates of the legacy WAR) but they should be manageable.
> > >
> > > A more complex version of the deploy time solution has the legacy web
> > > application placed in webapps, identified as a legacy webapp and then
> > > replaced with the new version (several ways to do this). The hard part
> > > here is the identification of the webapp as a Java EE app. The only
> > > reliable way to do this is class scanning and that is slow.
> > >
> > > The runtime approach converts classes and static resources as they are
> > > loaded. The classes are relatively easy to handle. A
> > > ClassFileTransformer can be added to the WebappClassLoader to do this.
> > > The static files are more of a problem. The good news is that the
> > > WebResources refactoring means static file access all goes through the
> > > same code but the filtering required essentially means we'd need to
> load
> > > static files into memory - regardless of size, convert them, and then
> > > update the cache with the converted version. That is likely to have a
> > > performance impact.
> > >
> > > Because of the performance impacts of handling static resources, the
> > > runtime approach also needs a way to identify applications that need
> > > conversion. I don't see a reliable, performant way to do that. Which, I
> > > think, leaves us with the simple deployment time approach and something
> > > (filename, special directory, something else) to mark an app as needing
> > > conversion.
> > >
> > > A final point, which probably should have been first, is is there a
> > > demand for this feature? Early indications from the users list and
> $work
> > > is that there is (going to be) a demand for this feature.
> > >
> > > Thoughts?
> > >
> >
> > We should also think about Tomcat Embedded! Many of the modern web
> > frameworks use embedded Tomcat/Jetty/Undertow.
> > I guess it should be even easier for them to set a property saying "I
> need
> > this app to be migrated" or to put it in the special folder
> > (webapps-legacy/).
> > We just need to make sure tomcat-embed is taken into account for the
> > eventual solution!
> >
>
> I'm actually not sure:
>
> 1. embed case almost never have a webapps/ folder and always uses flat
> classpath so only option is a javaagent but 2.
>

"almost never" does not mean that it is not possible to do
org.apache.catalina.startup.Tomcat#addContext("someName",
"/path/to/folder/with/static/files/which/need/to/be/migrated")
javaagent alone won't help you for any non-class files


> 2. embed case does not have this standalone status where you put an app in
> another one so if you want to change you change it in your build
>

I don't really understand you here


>
> I can see you speaking of spring-boot or so but this is all solved by maven
> shade plugin or gradle equivalent task so I wouldn't invest in a solution
> which - for this case - wouldnt be used compared to the standalone case
> which still has ops vs dev coverage and need IMHO.
>

Believe me it is not that simple.
Spring Boot Maven/Gradle plugins produce a jar file that contains other jar
files inside. Those jar files are not compressed for optimization reasons
and the Jakarta EE migration tool cannot handle that because
java.util.jar.* APIs does not provide a way to not compress, after
re-writing the classes/files.
So, a Spring Boot application cannot be really migrated to Jakarta EE.


>
>
> >
> > Martin
> >
> >
> > > Mark
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> > > For additional commands, e-mail: dev-h...@tomcat.apache.org
> > >
> > >
> >
>

Re: Integrating migration tool into Tomcat 10

[Romain Manni-Bucau]

Le mer. 10 févr. 2021 à 09:32, Martin Grigorov  a
écrit :

> Hi,
>
> On Tue, Feb 9, 2021 at 11:12 PM Mark Thomas  wrote:
>
> > Hi all,
> >
> > I've been looking at the options to integrate the Java EE to Jakarta EE
> > migration functionality into Tomcat 10.
> >
> > There are essentially two ways to do this: deployment time and runtime.
> >
> > The simplest solution to implement is probably a separate webapps-legacy
> > directory (or some other name) where WARs and DIRs added to that
> > directory get converted to Jakarta EE with the new version being placed
> > in the webapps directory. There are complexities (such as handling an
> > updates of the legacy WAR) but they should be manageable.
> >
> > A more complex version of the deploy time solution has the legacy web
> > application placed in webapps, identified as a legacy webapp and then
> > replaced with the new version (several ways to do this). The hard part
> > here is the identification of the webapp as a Java EE app. The only
> > reliable way to do this is class scanning and that is slow.
> >
> > The runtime approach converts classes and static resources as they are
> > loaded. The classes are relatively easy to handle. A
> > ClassFileTransformer can be added to the WebappClassLoader to do this.
> > The static files are more of a problem. The good news is that the
> > WebResources refactoring means static file access all goes through the
> > same code but the filtering required essentially means we'd need to load
> > static files into memory - regardless of size, convert them, and then
> > update the cache with the converted version. That is likely to have a
> > performance impact.
> >
> > Because of the performance impacts of handling static resources, the
> > runtime approach also needs a way to identify applications that need
> > conversion. I don't see a reliable, performant way to do that. Which, I
> > think, leaves us with the simple deployment time approach and something
> > (filename, special directory, something else) to mark an app as needing
> > conversion.
> >
> > A final point, which probably should have been first, is is there a
> > demand for this feature? Early indications from the users list and $work
> > is that there is (going to be) a demand for this feature.
> >
> > Thoughts?
> >
>
> We should also think about Tomcat Embedded! Many of the modern web
> frameworks use embedded Tomcat/Jetty/Undertow.
> I guess it should be even easier for them to set a property saying "I need
> this app to be migrated" or to put it in the special folder
> (webapps-legacy/).
> We just need to make sure tomcat-embed is taken into account for the
> eventual solution!
>

I'm actually not sure:

1. embed case almost never have a webapps/ folder and always uses flat
classpath so only option is a javaagent but 2.
2. embed case does not have this standalone status where you put an app in
another one so if you want to change you change it in your build

I can see you speaking of spring-boot or so but this is all solved by maven
shade plugin or gradle equivalent task so I wouldn't invest in a solution
which - for this case - wouldnt be used compared to the standalone case
which still has ops vs dev coverage and need IMHO.


>
> Martin
>
>
> > Mark
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> > For additional commands, e-mail: dev-h...@tomcat.apache.org
> >
> >
>

Re: Integrating migration tool into Tomcat 10

[Martin Grigorov]

Hi,

On Tue, Feb 9, 2021 at 11:12 PM Mark Thomas  wrote:

> Hi all,
>
> I've been looking at the options to integrate the Java EE to Jakarta EE
> migration functionality into Tomcat 10.
>
> There are essentially two ways to do this: deployment time and runtime.
>
> The simplest solution to implement is probably a separate webapps-legacy
> directory (or some other name) where WARs and DIRs added to that
> directory get converted to Jakarta EE with the new version being placed
> in the webapps directory. There are complexities (such as handling an
> updates of the legacy WAR) but they should be manageable.
>
> A more complex version of the deploy time solution has the legacy web
> application placed in webapps, identified as a legacy webapp and then
> replaced with the new version (several ways to do this). The hard part
> here is the identification of the webapp as a Java EE app. The only
> reliable way to do this is class scanning and that is slow.
>
> The runtime approach converts classes and static resources as they are
> loaded. The classes are relatively easy to handle. A
> ClassFileTransformer can be added to the WebappClassLoader to do this.
> The static files are more of a problem. The good news is that the
> WebResources refactoring means static file access all goes through the
> same code but the filtering required essentially means we'd need to load
> static files into memory - regardless of size, convert them, and then
> update the cache with the converted version. That is likely to have a
> performance impact.
>
> Because of the performance impacts of handling static resources, the
> runtime approach also needs a way to identify applications that need
> conversion. I don't see a reliable, performant way to do that. Which, I
> think, leaves us with the simple deployment time approach and something
> (filename, special directory, something else) to mark an app as needing
> conversion.
>
> A final point, which probably should have been first, is is there a
> demand for this feature? Early indications from the users list and $work
> is that there is (going to be) a demand for this feature.
>
> Thoughts?
>

We should also think about Tomcat Embedded! Many of the modern web
frameworks use embedded Tomcat/Jetty/Undertow.
I guess it should be even easier for them to set a property saying "I need
this app to be migrated" or to put it in the special folder
(webapps-legacy/).
We just need to make sure tomcat-embed is taken into account for the
eventual solution!

Martin


> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>

Re: Integrating migration tool into Tomcat 10

[Romain Manni-Bucau]

Hi,

I'd go simple:

1. Add in the tool jar a ClassFileTransformer
2. Make this transformer configurable (packages using "startsWith" which is
fast and optimizable - maybe something like
https://github.com/apache/openwebbeans-meecrowave/blob/master/meecrowave-core/src/main/java/org/apache/meecrowave/openwebbeans/KnownClassesFilter.java#L218
)
3. Add a transformer webresource - like in maven shade plugin - which is
about to transform a resource from a pattern (prefix/suffix cases being
optimized in terms of matching) and replacement

With these 3 new configurable classes tomcat stays pure and fast for all
cases but enables the migration on the fly when needed.
Can even be a tomcat-jakarta-migration-helper.jar totally independent of
tomcat - while it is released on central from time to time ;).

All mecanism requiring another webapp or a reload are promises of failures
IMHO since often webapps are dropped or undesired except the app webapps in
real or dev deployments.

Romain Manni-Bucau
@rmannibucau  |  Blog
 | Old Blog
 | Github  |
LinkedIn  | Book



Le mer. 10 févr. 2021 à 01:46, Emmanuel Bourg  a écrit :

> Le 2021-02-09 22:12, Mark Thomas a écrit :
>
> > Thoughts?
>
> I think this feature is really desirable. For the Debian packaging this
> would mean a faster transition to Tomcat 10.
>
> There are two issues:
> 1. How to identify a legacy application?
> 2. How to convert the application once identified?
>
> For the identification, either:
> a. separate the Java EE and Jakarta EE webapps in different directories.
> It's simple but it's an additional decision point for the user and there
> will be mistakes.
> b. keep the webapps in the same directory and scan the content. This
> degrades the performances for those having switched to Jakarta EE, so
> this should be optional, and probably disabled by default.
> c. keep the webapps in the same directory, load them as usual and
> trigger the conversion mechanism only if a class loading error related
> to the javax namespace occurs.
>
> Regarding the conversion, we can:
> a. Convert at runtime with class loading tricks, which is quite elegant
> but the price is paid every time the application is started.
> b. Convert at deployment time before loading the application, this means
> more file juggling but only the first start is impacted.
>
> The convenience of the automatic migration comes obviously at a price
> but I think it's important we ensure it doesn't impact those ready to
> jump to the new Jakarta EE world. For this reason I'm leaning toward the
> solution 1c + 2b (convert on errors, and convert the webapp before
> reloading).
>
> The start sequence would look like this:
> 1. foo.war is copied to the webapps directory by the user
> 2. The foo application is loaded
> 3. A NoClassDefFoundError regarding the javax.servlet.Servlet class is
> thrown, the migration mechanism is triggered
> 4. The migration tool is executed on foo.war. The original file is
> renamed foo.war.legacy and the migrated file remains as foo.war
> 5. The application is reloaded
>
> And on subsequent starts, the migrated application is loaded directly,
> with no performance impact.
>
> Emmanuel Bourg
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>

Re: Integrating migration tool into Tomcat 10

[Emmanuel Bourg]

Le 2021-02-09 22:12, Mark Thomas a écrit :


Thoughts?


I think this feature is really desirable. For the Debian packaging this 
would mean a faster transition to Tomcat 10.


There are two issues:
1. How to identify a legacy application?
2. How to convert the application once identified?

For the identification, either:
a. separate the Java EE and Jakarta EE webapps in different directories. 
It's simple but it's an additional decision point for the user and there 
will be mistakes.
b. keep the webapps in the same directory and scan the content. This 
degrades the performances for those having switched to Jakarta EE, so 
this should be optional, and probably disabled by default.
c. keep the webapps in the same directory, load them as usual and 
trigger the conversion mechanism only if a class loading error related 
to the javax namespace occurs.


Regarding the conversion, we can:
a. Convert at runtime with class loading tricks, which is quite elegant 
but the price is paid every time the application is started.
b. Convert at deployment time before loading the application, this means 
more file juggling but only the first start is impacted.


The convenience of the automatic migration comes obviously at a price 
but I think it's important we ensure it doesn't impact those ready to 
jump to the new Jakarta EE world. For this reason I'm leaning toward the 
solution 1c + 2b (convert on errors, and convert the webapp before 
reloading).


The start sequence would look like this:
1. foo.war is copied to the webapps directory by the user
2. The foo application is loaded
3. A NoClassDefFoundError regarding the javax.servlet.Servlet class is 
thrown, the migration mechanism is triggered
4. The migration tool is executed on foo.war. The original file is 
renamed foo.war.legacy and the migrated file remains as foo.war

5. The application is reloaded

And on subsequent starts, the migrated application is loaded directly, 
with no performance impact.


Emmanuel Bourg

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Integrating migration tool into Tomcat 10

[Filip Hanik]

On Tue, Feb 9, 2021 at 14:34 Rémy Maucherat  wrote:

> On Tue, Feb 9, 2021 at 10:12 PM Mark Thomas  wrote:
>
> > Hi all,
> >
> > I've been looking at the options to integrate the Java EE to Jakarta EE
> > migration functionality into Tomcat 10.
> >
> > There are essentially two ways to do this: deployment time and runtime.
> >
> > The simplest solution to implement is probably a separate webapps-legacy
> > directory (or some other name) where WARs and DIRs added to that
> > directory get converted to Jakarta EE with the new version being placed
> > in the webapps directory. There are complexities (such as handling an
> > updates of the legacy WAR) but they should be manageable.
> >
> > A more complex version of the deploy time solution has the legacy web
> > application placed in webapps, identified as a legacy webapp and then
> > replaced with the new version (several ways to do this). The hard part
> > here is the identification of the webapp as a Java EE app. The only
> > reliable way to do this is class scanning and that is slow.
> >
> > The runtime approach converts classes and static resources as they are
> > loaded. The classes are relatively easy to handle. A
> > ClassFileTransformer can be added to the WebappClassLoader to do this.
> > The static files are more of a problem. The good news is that the
> > WebResources refactoring means static file access all goes through the
> > same code but the filtering required essentially means we'd need to load
> > static files into memory - regardless of size, convert them, and then
> > update the cache with the converted version. That is likely to have a
> > performance impact.
> >
> > Because of the performance impacts of handling static resources, the
> > runtime approach also needs a way to identify applications that need
> > conversion. I don't see a reliable, performant way to do that. Which, I
> > think, leaves us with the simple deployment time approach and something
> > (filename, special directory, something else) to mark an app as needing
> > conversion.
> >
> > A final point, which probably should have been first, is is there a
> > demand for this feature? Early indications from the users list and $work
> > is that there is (going to be) a demand for this feature.
> >
> > Thoughts?
> >
>
> I kind of proposed the "simplest" option [the one using a separate appBase
> for the EE 8 and earlier webapps] a couple times. A slightly more complex
> deploy time option could look more polished and maybe preferable, like that
> "filename" idea maybe ? Or a marker file ? I'm not sure.
>
> There's a demand for the feature for sure, but probably only up to the
> point people realize it's not actually that helpful. This adds a step to
> deployment which may fail. The offline tool however is more efficient and
> safer. Also the tool now has quite a few "advanced" options [the ones you
> just added] for migrating trickier webapps, so how is an automatic
> migration with defaults going to work out ? This looks like asking people
> to fill up the BZ with stuff.
>
> So assuming the feature goes in, maybe a hybrid solution could work better
> than a pure runtime or deploy time solution. The classloader classes and
> resources could rely on a runtime conversion [a bit costly but probably
> safer], while the static resources could be converted at deployment time
> [it's hard to give decent guarantees on resource use if done at runtime].
>
> Rémy


We’ve spent two decades optimizing the run time within the constraints that
the Servlet spec has given to us. This was driven by the market, and
accentuated when containerization became a thing.

I would suggest that those optimization not be lost and that the extra time
spent at startup, a highly repetitive operation these days, would be
something the user would have to opt in to.

Personally I like a complete offline model, yet there is immense value for
organizations that have stale projects to be upgraded easily. (albeit, one
can argue they can stay on v9 for a very long time too)

>
>
> >
> > Mark
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> > For additional commands, e-mail: dev-h...@tomcat.apache.org
> >
> >
>

Re: Integrating migration tool into Tomcat 10

[Rémy Maucherat]

On Tue, Feb 9, 2021 at 10:12 PM Mark Thomas  wrote:

> Hi all,
>
> I've been looking at the options to integrate the Java EE to Jakarta EE
> migration functionality into Tomcat 10.
>
> There are essentially two ways to do this: deployment time and runtime.
>
> The simplest solution to implement is probably a separate webapps-legacy
> directory (or some other name) where WARs and DIRs added to that
> directory get converted to Jakarta EE with the new version being placed
> in the webapps directory. There are complexities (such as handling an
> updates of the legacy WAR) but they should be manageable.
>
> A more complex version of the deploy time solution has the legacy web
> application placed in webapps, identified as a legacy webapp and then
> replaced with the new version (several ways to do this). The hard part
> here is the identification of the webapp as a Java EE app. The only
> reliable way to do this is class scanning and that is slow.
>
> The runtime approach converts classes and static resources as they are
> loaded. The classes are relatively easy to handle. A
> ClassFileTransformer can be added to the WebappClassLoader to do this.
> The static files are more of a problem. The good news is that the
> WebResources refactoring means static file access all goes through the
> same code but the filtering required essentially means we'd need to load
> static files into memory - regardless of size, convert them, and then
> update the cache with the converted version. That is likely to have a
> performance impact.
>
> Because of the performance impacts of handling static resources, the
> runtime approach also needs a way to identify applications that need
> conversion. I don't see a reliable, performant way to do that. Which, I
> think, leaves us with the simple deployment time approach and something
> (filename, special directory, something else) to mark an app as needing
> conversion.
>
> A final point, which probably should have been first, is is there a
> demand for this feature? Early indications from the users list and $work
> is that there is (going to be) a demand for this feature.
>
> Thoughts?
>

I kind of proposed the "simplest" option [the one using a separate appBase
for the EE 8 and earlier webapps] a couple times. A slightly more complex
deploy time option could look more polished and maybe preferable, like that
"filename" idea maybe ? Or a marker file ? I'm not sure.

There's a demand for the feature for sure, but probably only up to the
point people realize it's not actually that helpful. This adds a step to
deployment which may fail. The offline tool however is more efficient and
safer. Also the tool now has quite a few "advanced" options [the ones you
just added] for migrating trickier webapps, so how is an automatic
migration with defaults going to work out ? This looks like asking people
to fill up the BZ with stuff.

So assuming the feature goes in, maybe a hybrid solution could work better
than a pure runtime or deploy time solution. The classloader classes and
resources could rely on a runtime conversion [a bit costly but probably
safer], while the static resources could be converted at deployment time
[it's hard to give decent guarantees on resource use if done at runtime].

Rémy

>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>

Integrating migration tool into Tomcat 10

[Mark Thomas]

Hi all,

I've been looking at the options to integrate the Java EE to Jakarta EE
migration functionality into Tomcat 10.

There are essentially two ways to do this: deployment time and runtime.

The simplest solution to implement is probably a separate webapps-legacy
directory (or some other name) where WARs and DIRs added to that
directory get converted to Jakarta EE with the new version being placed
in the webapps directory. There are complexities (such as handling an
updates of the legacy WAR) but they should be manageable.

A more complex version of the deploy time solution has the legacy web
application placed in webapps, identified as a legacy webapp and then
replaced with the new version (several ways to do this). The hard part
here is the identification of the webapp as a Java EE app. The only
reliable way to do this is class scanning and that is slow.

The runtime approach converts classes and static resources as they are
loaded. The classes are relatively easy to handle. A
ClassFileTransformer can be added to the WebappClassLoader to do this.
The static files are more of a problem. The good news is that the
WebResources refactoring means static file access all goes through the
same code but the filtering required essentially means we'd need to load
static files into memory - regardless of size, convert them, and then
update the cache with the converted version. That is likely to have a
performance impact.

Because of the performance impacts of handling static resources, the
runtime approach also needs a way to identify applications that need
conversion. I don't see a reliable, performant way to do that. Which, I
think, leaves us with the simple deployment time approach and something
(filename, special directory, something else) to mark an app as needing
conversion.

A final point, which probably should have been first, is is there a
demand for this feature? Early indications from the users list and $work
is that there is (going to be) a demand for this feature.

Thoughts?

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org