Top-posting as cutting snippets out of the response I think paraphrases it inaccurately -- it's a good email.
Net I agree with most points including the bottomline comment that there will definitely be edge cases we will not be able to support with the Eclipse Transformer -- namely patching in new codeblocks into existing bytecode. That on top of stomping out the bugs means we'd be unlikely to get to the end of this anytime soon. I've gone ahead and created us a new repo and seeded it with the bytecode scanning/transformation code that I've been working on to Scan/Transform Maven Central. - https://github.com/apache/tomee-patch-plugin/commit/2a06d9f081c1db05164286db9149ba17c30754dc The goal in my mind is that we have something into which we can put all the dirty hacks we want and be as unrestricted as possible to solve this problem as quickly and thoroughly as possible. I don't think this is code want to own long-term and any improvements should go into a mix of the Eclipse Transformer, Jkta or a more general-purposed location. But at least for the moment we'll have all the control we need to patch our zips using whatever custom code without having to be held up by what the "proper" abstraction is or any abstraction at all. We just get it to work and move on. I'm confident we can stomp out our edge cases quickly this way and may even have a perfectly running server under the jakarta namespace in time for some kind of release next week (if we want to do a release). -- David Blevins http://twitter.com/dblevins http://www.tomitribe.com > On Jun 10, 2020, at 2:11 AM, Jonathan Gallimore > <[email protected]> wrote: > > In terms of the technical aspects of doing the find/replace on non-class > files with the Transformer - its possible - it would require a (not huge) > change, but its possible. There are some specific cases that the > transformer handles already, such as XML schemas (and we have nothing > configured for that, so that wouldn't be a surprise if work is needed > there) and service loaders (which should be working ok). There may be other > challenges - one that I had to solve already was with servlet-api jar, > which contains properties files for localized strings, which are loaded as > resources from the classpath. These sat under javax - here's an example: > https://github.com/apache/tomcat/blob/9.0.x/java/javax/servlet/LocalStrings.properties. > The transformer rewrote the references in the code to jakarta.servlet > (great!) but didn't move the properties file - it left it under javax. I > solved that by adding another action to the transformer to deal with that: > https://github.com/tomitribe/transformer/blob/master/transformer/src/main/java/org/eclipse/transformer/action/impl/RelocateResourceActionImpl.java > . > > I'm not particularly surprised that there are edge cases that we can't > transform - I was expecting that, particularly in class loading logic where > there are matches on things like "javax.". I think there's a natural > temptation to add a rule which is simply "javax" -> "jakarta", but that > will likely break as much as it fixes as there are exceptions to the rule > with some stuff staying behind in javax. The issue I highlighted in > Eclipselink looks like a case we simply won't be able to transform without > a significant amount of complexity. Upgrading to a version where that is > fixed breaks the TomEE build entirely, as EclipseLink is then expecting > everything to be under jakarta. One option there might be to do a reverse > transformation on that one jar, add that to the build, and then include it > in the transformation to jakarta. I don't know if that sounds ridiculous > written down, but it certainly does in my head. I think we can simply grab > the current EclipseLink jar, and patch it slightly so it then transforms > nicely. We've patched various jars including JSTL and OpenJPA in the past. > I suspect I'll have this done and checked in today. > > Shooting for phase 1 perfection I think is tricky. Partly because when > doing this type of work, there are always edge cases that are hard to find > and fix, and partly because even if we achieved "perfection", there's still > no guarantee it would actually work. There are still replacements that > would break, and then there are other crazy issues like signed jars that > break the second you touch them (ecj, for example). > > I personally have already started booting the server, and actually trying > stuff. Moviefun is a fun example, as we have versions of it that cover > Servlets, JSPs, EJBs. JPA, and JAX-RS. I imagine testing some basic JAX-WS > wouldn't be a stretch. I uncovered the EclipseLink issue by actually > trying to run moviefun on the server, by hand. If we perfectly achieved the > phase 1 goal, we'd still have missed this reference, because the text > "javax" doesn't feature in that block of code. The references there are > "kotlin", "java", "x", and "persistence". So we'd still have needed to boot > the server, and try it. > > I also have applications that test MVC, and connectors as well. They also > use Hibernate.... which could be interesting. > > Short answer (and I understand if the above is too long!) is I'd like to > see the 6053 hits, along with where they are, and we can discuss whether > they need to be transformed, and if so, what rules we might apply in the > transformer. Additionally, I think we're at the point where we ought to > push some binaries and with the disclaimer that we know there's a good > chance that lots of things might not work, but please can people kick the > tyres anyway and report back. If we try and achieve phase 1 perfection, I > fear phase 2 and actually trying the server are potentially a long way off. > > Jon > > > > On Wed, Jun 10, 2020 at 2:22 AM David Blevins <[email protected]> > wrote: > >>> On Jun 9, 2020, at 2:45 PM, David Blevins <[email protected]> >> wrote: >>> >>>> On Jun 6, 2020, at 3:26 PM, David Blevins <[email protected]> >> wrote: >>>> >>>> All total about 2508 hits. >>>> >>> >>> With latest changes we're down to about 672 hits. >>> >>> - >> https://github.com/dblevins/tomee-analysis/blob/6ee78178aa5dd55447943fd25cd5821f6159fc9e/README.adoc >>> >> >> Ok, I expanded the search for potentially missed "javax" references to now >> also search non class files. This gives us a few thousand more hits (6053 >> total). Of them, 4980 are for javax.faces most across .tld, .js and .xsd >> files. >> >> At this point, it's a little unclear how to proceed. There are actually >> two problems: >> >> - Simple find/replace on all non-class files in a jar >> - Doing it elegantly not offending the design of the Eclipse Transformer >> >> The trick I'm having is I personally have a tendency to code like this: >> >> Phase 1: code till the problem is completely solved including each and >> every edge case. Avoid as much design overhead as possible, at this point >> they're just handcuffs. >> >> Phase 2: once phase 1 is fully complete, take stock of the code and >> redesign it with the whole picture in mind. >> >> I feel like where we are at with the Eclipse Transformer is it's already >> firming up the design and we're not out of phase 1 yet. I could be wrong >> on that. >> >> My fear is that we spend the next few weeks feeding edge-cases in one at a >> time, each time treating the design as perfect trying to make as few design >> changes as possible, sometimes going a little out of our way to make that >> happen and this drags on. >> >> Alternatively, we could just get to the end of this in a hacked fashion, >> get the server to boot and completely run, and *then* see what changes we'd >> need to make to the Eclipse Transformer design to get it to do what we need. >> >> Jon, you're the only one with experience with the Eclipse Transformer, I'm >> very curious what you think. >> >> >> -David >> >> >> >> >> >> >>
smime.p7s
Description: S/MIME cryptographic signature
